• Stars
    star
    2,283
  • Rank 20,175 (Top 0.4 %)
  • Language
    Go
  • License
    MIT License
  • Created over 7 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

CLI for managing secrets

Chamber

Chamber is a tool for managing secrets. Currently it does so by storing secrets in SSM Parameter Store, an AWS service for storing secrets.

For detailed info about using chamber, please read The Right Way To Manage Secrets

v2.0 Breaking Changes

Starting with version 2.0, chamber uses parameter store's path based API by default. Chamber pre-2.0 supported this API using the CHAMBER_USE_PATHS environment variable. The paths based API has performance benefits and is the recommended best practice by AWS.

As a side effect of this change, if you didn't use path based secrets before 2.0, you will need to set CHAMBER_NO_PATHS to enable the old behavior. This option is deprecated, and We recommend only using this setting for supporting existing applications.

To migrate to the new format, you can take advantage of the export and import commands. For example, if you wanted to convert secrets for service foo to the new format using chamber 2.0, you can do:

CHAMBER_NO_PATHS=1 chamber export foo | chamber import foo -

v2.13.0 Breaking Changes

Support for very old versions of Go has been dropped, and chamber will only test against versions of Go covered by the Go Release Policy, e.g. the two most recent major versions. This will ensure that we can reliably update dependencies as needed. Additionally, chamber binaries will be built with the latest stable version of Go at the time of release.

Installing

If you have a functional go environment, you can install with:

go install github.com/segmentio/chamber/v2@latest

Caveat About chamber version and go install

Note that installing with go install will not produce an executable containing any versioning information. This information is passed at compilation time when the Makefile is used for compilation. Without this information, chamber version outputs the following:

$ chamber version
chamber dev

See the wiki for more installation options like Docker images, Linux packages, and precompiled binaries.

Authenticating

Using chamber requires you to be running in an environment with an authenticated AWS user which has the appropriate permission to read/write values to SSM Parameter Store.

This is going to vary based on your organization but chamber needs AWS credentials to run.

One of the easiest ways to do so is by using aws-vault. To adjust these instructions for your needs, examine the env output of Aws-Vault: How It Works and use your organization's secrets tool accordingly with chamber.

An aws-vault Usage Example With Chamber

aws-vault exec prod -- chamber

For this reason, it is recommended that you create an alias in your shell of choice to save yourself some typing, for example (from my .zshrc):

alias chamberprod='aws-vault exec production -- chamber'

Setting Up KMS

Chamber expects to find a KMS key with alias parameter_store_key in the account that you are writing/reading secrets. You can follow the AWS KMS documentation to create your key, and follow this guide to set up your alias.

If you are a Terraform user, you can create your key with the following:

resource "aws_kms_key" "parameter_store" {
  description             = "Parameter store kms master key"
  deletion_window_in_days = 10
  enable_key_rotation     = true
}

resource "aws_kms_alias" "parameter_store_alias" {
  name          = "alias/parameter_store_key"
  target_key_id = "${aws_kms_key.parameter_store.id}"
}

If you'd like to use an alternate KMS key to encrypt your secrets, you can set the environment variable CHAMBER_KMS_KEY_ALIAS. As an example, the following will use your account's default SSM alias: CHAMBER_KMS_KEY_ALIAS=aws/ssm

Usage

Writing Secrets

$ chamber write <service> <key> <value|->

This operation will write a secret into the secret store. If a secret with that key already exists, it will increment the version and store a new value.

If - is provided as the value argument, the value will be read from standard input.

Secret keys are normalized automatically. The - will be _ and the letters will be converted to upper case (for example a secret with key secret_key and secret-key will become SECRET_KEY).

Listing Secrets

$ chamber list service
Key         Version                  LastModified      User
apikey      2                        06-09 17:30:56    daniel-fuentes
other       1                        06-09 17:30:34    daniel-fuentes

Listing secrets should show the key names for a given service, along with other useful metadata including when the secret was last modified, who modified it, and what the current version is.

$ chamber list -e service
Key         Version                  LastModified      User             Value
apikey      2                        06-09 17:30:56    daniel-fuentes   apikeyvalue
other       1                        06-09 17:30:34    daniel-fuentes   othervalue

Listing secrets with expand parameter should show the key names and values for a given service, along with other useful metadata including when the secret was last modified, who modified it, and what the current version is.

Historic view

$ chamber history service key
Event       Version     Date            User
Created     1           06-09 17:30:19  daniel-fuentes
Updated     2           06-09 17:30:56  daniel-fuentes

The history command gives a historical view of a given secret. This view is useful for auditing changes, and can point you toward the user who made the change so it's easier to find out why changes were made.

Exec

$ chamber exec <service...> -- <your executable>

exec populates the environment with the secrets from the specified services and executes the given command. Secret keys are converted to upper case (for example a secret with key secret_key will become SECRET_KEY).

Secrets from services are loaded in the order specified in the command. For example, if you do chamber exec app apptwo -- ... and both apps have a secret named api_key, the api_key from apptwo will be the one set in your environment.

Reading

$ chamber read service key
Key             Value                           Version         LastModified    User
key             secret                          1               06-09 17:30:56  daniel-fuentes

read provides the ability to print out the value of a single secret, as well as the secret's additional metadata. It does not provide the ability to print out multiple secrets in order to discourage accessing extra secret material that is unneeded. Parameter store automatically versions secrets and passing the --version/-v flag to read can print older versions of the secret. Default version (-1) is the latest secret.

Exporting

$ chamber export [--format <format>] [--output-file <file>]  <service...>
{"key":"secret"}

export provides ability to export secrets in various file formats. The following file formats are supported:

  • json (default)
  • yaml
  • java-properties
  • csv
  • tsv
  • dotenv
  • tfvars

File is written to standard output by default but you may specify an output file.

Caveat About Environment Variables

chamber can emit environment variables in both dotenv format and exported shell environment variables. As chamber allows creating key names that are themselves not valid shell variable names, secrets emitted in this format will have their keys modified to confirm to POSIX shell environment variable naming rules:

  • variable names must begin with a letter or an underscore
    • variable names must not begin with a number
  • variable names must only contain letters, numbers, or underscores

Notes About Dotenv Format

As there is no formal dotenv spec, chamber attempts to adhere to compliance with joho/godotenv (which is itself a port of the Ruby library bkeepers/dotenv). The output should be generally cross-compatible with alternative parsers, but without a formal spec compatibility is not guaranteed.

Of note:

  • all key names will be sanitized according the the POSIX shell rules above, and cast to uppercase
  • all values will be rendered using special characters instead of string literals, e.g. newlines replaced with the character \n, tabstops replaced with the character \t, etc.
    • no whitespace trimming will be performed on any values

Notes About Exported Environment Variables

Alternatively, chamber may be used to set local environment variables directly with the chamber env command. For example,

source <(chamber env service)`
printf "%s" "$SERVICE_VAR"

Note that all secrets printed this way will be prefixed with export, so if sourced inline as in the above example, then any and all secrets will then be available to any process run after sourcing.

the env subcommand supports output formatting in two specific ways:

chamber env -h
Print the secrets from the parameter store in a format to export as environment variables

Usage:
  chamber env <service> [flags]

Flags:
  -p, --preserve-case    preserve variable name case
  -e, --escape-strings   escape special characters in values

As chamber allows creation of keys with mixed case, --preserve-case will ensure that the original key case is preserved. Note that this will not prevent the key name from being sanitized according to the above POSIX shell rules. By default, values will be rendered using string literals, e.g. newlines will be printed as literal newlines, tabstops as literal tabstops. Output may be emitted using escaped special characters instead (identical to chamber export -o dotenv)) by using the flag --escape-strings.

Importing

$ chamber import [--normalize-keys] <service> <filepath>

import provides the ability to import secrets from a json or yaml file (like the kind you get from chamber export).

Note By default, import will not normalize key inputs, meaning that keys will be written to the secrets backend in the format they exist in the source file. In order to normalize keys on import, provide the --normalize-keys flag

When normalizing keys, before write, the key will be be first converted to lowercase to match how chamber write handles keys.

Example: DB_HOST will be converted to db_host.

You can set filepath to - to instead read input from stdin.

Deleting

$ chamber delete [--exact-key] service key

delete provides the ability to remove a secret from chamber permanently, including the secret's additional metadata. There is no way to recover a secret once it has been deleted so care should be taken with this command.

Note By default, delete will normalize any provided keys. To change that behavior, provide the --exact-key flag to attempt to delete the raw provided key.

Example: Given the following setup,

$ chamber list service
Key         Version                  LastModified      User
apikey      2                        06-09 17:30:56    daniel-fuentes
APIKEY      1                        06-09 17:30:34    daniel-fuentes

Calling

$ chamber delete --exact-key service APIKEY

will delete only APIKEY from the service and leave only

$ chamber list service
Key         Version                  LastModified      User
apikey      2                        06-09 17:30:56    daniel-fuentes

Finding

$ chamber find key

find provides the ability to locate which services use the same key names.

$ chamber find value --by-value

Passing --by-value or -v will search the values of all secrets and return the services and keys which match.

AWS Region

Chamber uses AWS SDK for Go. To use a region other than what is specified in $HOME/.aws/config, set the environment variable "AWS_REGION".

$ AWS_REGION=us-west-2 chamber list service
Key         Version                  LastModified      User
apikey      3                        07-10 09:30:41    daniel-fuentes
other       1                        07-10 09:30:35    daniel-fuentes

Chamber does not currently read the value of "AWS_DEFAULT_REGION". See https://github.com/aws/aws-sdk-go#configuring-aws-region for more details.

If you'd like to use a different region for chamber without changing AWS_REGION, you can use CHAMBER_AWS_REGION to override just for chamber.

Custom SSM Endpoint

If you'd like to use a custom SSM endpoint for chamber, you can use CHAMBER_AWS_SSM_ENDPOINT to override AWS default URL.

S3 Backend (Experimental)

By default, chamber store secrets in AWS Parameter Store. We now also provide an experimental S3 backend for storing secrets in S3 instead.

To configure chamber to use the S3 backend, use chamber -b s3 --backend-s3-bucket=mybucket. Preferably, this bucket should reject uploads that do not set the server side encryption header (see this doc for details how)

This feature is experimental, and not currently meant for production work.

S3 Backend using KMS Key Encryption (Experimental)

This backend is similar to the S3 Backend but uses KMS Key Encryption to encrypt your documents at rest, similar to the SSM Backend which encrypts your secrets at rest. You can read how S3 Encrypts documents with KMS here.

The highlights of SSE-KMS are:

  • You can choose to create and manage encryption keys yourself, or you can choose to use your default service key uniquely generated on a customer by service by region level.
  • The ETag in the response is not the MD5 of the object data.
  • The data keys used to encrypt your data are also encrypted and stored alongside the data they protect.
  • Auditable master keys can be created, rotated, and disabled from the AWS KMS console.
  • The security controls in AWS KMS can help you meet encryption-related compliance requirements.

Source: Protecting data using server-side encryption with AWS Key Management Service keys (SSE-KMS)

To configure chamber to use the S3 KMS backend, use chamber -b s3-kms --backend-s3-bucket=mybucket --kms-key-alias=alias/keyname. You must also supply an environment variable of the KMS Key Alias to use CHAMBER_KMS_KEY_ALIAS, by default "alias/parameter_store_key" will be used.

Preferably, this bucket should reject uploads that do not set the server side encryption header (see this doc for details how)

When changing secrets between KMS Keys, you must first delete the Chamber secret with the existing KMS Key, then write it again with new KMS Key.

If services contain multiple KMS Keys, chamber list and chamber exec will only show Chamber secrets encrypted with KMS Keys you have access to.

This feature is experimental, and not currently meant for production work.

Null Backend (Experimental)

If it's preferred to not use any backend at all, use chamber -b null. Doing so will forward existing ENV variables as if Chamber is not in between.

This feature is experimental, and not currently meant for production work.

Analytics

chamber includes some usage analytics code which Segment uses internally for tracking usage of internal tools. This analytics code is turned off by default, and can only be enabled via a linker flag at build time, which we do not set for public github releases.

Releasing

To cut a new release, just push a tag named v<semver> where <semver> is a valid semver version. This tag will be used by Github Actions to automatically publish a github release.

More Repositories

1

evergreen

🌲 Evergreen React UI Framework by Segment
JavaScript
12,161
star
2

kafka-go

Kafka library in Go
Go
7,518
star
3

analytics.js

The hassle-free way to integrate analytics into any web application.
JavaScript
4,775
star
4

myth

A CSS preprocessor that acts like a polyfill for future versions of the spec.
JavaScript
4,345
star
5

ksuid

K-Sortable Globally Unique IDs
Go
4,121
star
6

daydream

A chrome extension to record your actions into a nightmare or puppeteer script
JavaScript
2,768
star
7

stack

A set of Terraform modules for configuring production infrastructure with AWS
HCL
2,098
star
8

ui-box

Blazing Fast React UI Primitive
TypeScript
1,052
star
9

encoding

Go package containing implementations of efficient encoding, decoding, and validation APIs.
Go
911
star
10

golines

A golang formatter that fixes long lines
Go
803
star
11

asm

Go library providing algorithms optimized to leverage the characteristics of modern CPUs
Go
795
star
12

analytics-node

The hassle-free way to integrate analytics into any node application.
JavaScript
593
star
13

topicctl

Tool for declarative management of Kafka topics
Go
558
star
14

aws-okta

aws-vault like tool for Okta authentication
Go
541
star
15

niffy

Perceptual diffing suite built on Nightmare
JavaScript
535
star
16

analytics-ios

The hassle-free way to integrate analytics into any iOS application.
Objective-C
388
star
17

analytics-ruby

The hassle-free way to integrate analytics into any Ruby application.
Ruby
374
star
18

analytics-android

The hassle-free way to add analytics to your Android app.
Java
373
star
19

analytics-react-native

The hassle-free way to add analytics to your React-Native app.
TypeScript
337
star
20

consent-manager

Drop-in consent management plugin for analytics.js
TypeScript
326
star
21

parquet-go

Go library to read/write Parquet files
Go
314
star
22

ts-mysql-plugin

A typescript language service plugin that gives superpowers to SQL tagged template literals.
TypeScript
312
star
23

analytics-next

Segment Analytics.js 2.0
TypeScript
294
star
24

specs

Peer into your ECS clusters
JavaScript
273
star
25

fasthash

Go package porting the standard hashing algorithms to a more efficient implementation.
Go
261
star
26

ctlstore

Control Data Store
Go
261
star
27

ware

Easily create your own middleware layer.
JavaScript
254
star
28

analytics-php

The hassle-free way to integrate analytics into any php application.
PHP
252
star
29

analytics-python

The hassle-free way to integrate analytics into any python application.
Python
231
star
30

chrome-sidebar

Easiest way to embed an iframe as a chrome extension
JavaScript
208
star
31

typewriter

Type safety + intellisense for your Segment analytics
TypeScript
206
star
32

nsq.js

NSQ client for nodejs
JavaScript
203
star
33

stats

Go package for abstracting stats collection
Go
202
star
34

threat-modeling-training

Segment's Threat Modeling training for our engineers
197
star
35

in-eu

🇪🇺 privacy first EU detection library for browsers
JavaScript
180
star
36

kubectl-curl

Kubectl plugin to run curl commands against kubernetes pods
Go
167
star
37

go-prompt

Go terminal prompts.
Go
167
star
38

analytics-react

[DEPRECATED AND UNSUPPORTED] The hassle-free way to integrate analytics into your React application.
JavaScript
160
star
39

is-url

Loosely validate a URL.
JavaScript
160
star
40

cwlogs

CLI tool for reading logs from Cloudwatch Logs
Go
142
star
41

kubeapply

A lightweight tool for git-based management of Kubernetes configs
Go
141
star
42

analytics-go

Segment analytics client for Go
Go
136
star
43

analytics.js-core

The hassle-free way to integrate analytics into any web application.
TypeScript
132
star
44

dependency-report

Generate usage reports of your JS dependencies
JavaScript
129
star
45

ecs-logs

Log forwarder for services ran by ecs-agent.
Go
115
star
46

analytics-java

The hassle-free way to integrate analytics into any java application.
Java
113
star
47

analytics.js-integrations

Monorepo housing Segment's analytics.js integrations
JavaScript
112
star
48

go-athena

Golang database/sql driver for AWS Athena
Go
107
star
49

Analytics.NET

The hassle-free way to integrate analytics into any C# / .NET application.
C#
107
star
50

go-queue

NSQ consumer convenience layer.
Go
104
star
51

analytics-swift

The hassle-free way to add Segment analytics to your Swift app (iOS/tvOS/watchOS/macOS/Linux).
Swift
102
star
52

xml-parser

simple non-compliant xml parser for nodejs
JavaScript
101
star
53

backo

exponential backoff without the weird cruft
JavaScript
99
star
54

analytics-vue

The hassle-free way to integrate analytics into your Vue application.
Vue
98
star
55

nsq-go

Go package providing tools for building NSQ clients, servers and middleware.
Go
94
star
56

consul-go

Go package providing building blocks for interacting with Consul.
Go
90
star
57

frictionless-signup

Reduce friction and increase customer data in your online forms using Segment & Clearbit
JavaScript
86
star
58

superagent-retry

Retry superagent requests for common hangups
JavaScript
85
star
59

pg-escape

sprintf-style postgres query escaping and helper functions
JavaScript
84
star
60

conf

Go package for loading program configuration from multiple sources.
Go
81
star
61

orbital

🚀🌏 A simple end-to-end testing framework for Go
Go
80
star
62

functions-library

A library of example functions to use with the Segment Developer Center
JavaScript
75
star
63

inbound

A url and referrer parsing library for node.
JavaScript
72
star
64

decibel

A small iOS app for recording office noise dB levels to Datadog.
Swift
69
star
65

analytics-angular

The hassle-free way to integrate analytics into your Angular application.
TypeScript
68
star
66

events

Go package for routing, formatting and publishing events produced by a program.
Go
62
star
67

glue

Generate typed Golang RPC clients from server code
Go
60
star
68

pingdummy

Example application for segmentio/stack
JavaScript
60
star
69

go-loggly

Loggly client for Go
Go
59
star
70

analytics-rust

Segment analytics client for Rust
Rust
55
star
71

retrofit-jsonrpc

Json-RPC with Retrofit.
Java
54
star
72

snippet

Render the analytics.js snippet.
JavaScript
53
star
73

nsq_to_redis

NSQ ✈ Redis {pubsub, capped lists}
Go
52
star
74

segment-proxy

Proxies requests to the Segment CDN and Tracking API.
Go
51
star
75

statsy

Simple statsd client for nodejs
JavaScript
49
star
76

sherlock

A pluggable service-detection tool
JavaScript
49
star
77

is-email

Component: loosely validate an email address.
JavaScript
49
star
78

objconv

A Go package exposing encoder and decoders that support data streaming to and from multiple formats.
Go
49
star
79

cli

Go package providing high-level constructs for command-line tools.
Go
48
star
80

facade

Providing common fields for analytics integrations, since 2013.
JavaScript
47
star
81

agecache

An LRU cache with support for max age
Go
47
star
82

validate-form

Easily validate a form element against a set of rules.
JavaScript
44
star
83

go-stats

Go stats ticker utility
Go
44
star
84

go-snakecase

Faster snakecase implementation
Go
43
star
85

utm-params

parse and get all utm parameters
JavaScript
42
star
86

aws-billing

An API to learn how much your AWS hosting costs every month
JavaScript
39
star
87

action-destinations

Action Destinations are the new way to build streaming destinations on Segment.
TypeScript
38
star
88

testdemo

Examples for https://segment.com/blog/5-advanced-testing-techniques-in-go/
Go
38
star
89

data-digger

Dig through structured messages in Kafka, S3, or local files
Go
37
star
90

segment-docs

Segment Documentation. Powered by Jekyll.
HTML
36
star
91

feature

Feature gate database designed for simplicity and efficiency.
Go
36
star
92

redis-go

Go package providing tools for building redis clients, servers and middleware.
Go
36
star
93

http_to_nsq

Publishes HTTP requests to NSQD (for CI webhooks etc)
Go
36
star
94

analytics.js-integration

The base integration factory used to create custom analytics integrations for analytics.js.
JavaScript
35
star
95

ebs-backup

Backup EBS Volumes
Go
34
star
96

Analytics.Xamarin

Analytics for Xamarin, a portable class library supporting iOS, Android, Mac OS, and others.
C#
34
star
97

go-hll

Go implementation of HLL that plays nicely with other languages
Go
34
star
98

terraform-segment-data-lakes

Terraform modules which create AWS resources for a Segment Data Lake.
HCL
34
star
99

analytics-kotlin

The hassle-free way to add Segment analytics to your Kotlin app (Android/JVM).
Kotlin
32
star
100

errors-go

Go package providing various error handling primitives.
Go
32
star