Blog Post
Kubernetes and Secrets Management in the Cloud
secrets-init
secrets-init is a minimalistic init system designed to run as PID 1 inside container environments, similar to dumb-init, integrated with multiple secrets manager services:
Why you need an init system
Please read Yelp dumb-init repo explanation
Summary:
- Proper signal forwarding
- Orphaned zombies reaping
What secrets-init does
secrets-init runs as PID 1, acting like a simple init system. It launches a single process and then proxies all received signals to a session rooted at that child process.
secrets-init also passes almost all environment variables without modification, replacing secret variables with values from secret management services.
Integration with AWS Secrets Manager
User can put AWS secret ARN as environment variable value. The secrets-init will resolve any environment value, using specified ARN, to referenced secret value.
If the secret is saved as a Key/Value pair, all the keys are applied to as environment variables and passed. The environment variable passed is ignored unless it is inside the key/value pair.
# environment variable passed to `secrets-init`
MY_DB_PASSWORD=arn:aws:secretsmanager:$AWS_REGION:$AWS_ACCOUNT_ID:secret:mydbpassword-cdma3
# environment variable passed to child process, resolved by `secrets-init`
MY_DB_PASSWORD=very-secret-passwordIntegration with AWS Systems Manager Parameter Store
It is possible to use AWS Systems Manager Parameter Store to store application parameters and secrets.
User can put AWS Parameter Store ARN as environment variable value. The secrets-init will resolve any environment value, using specified ARN, to referenced parameter value.
# environment variable passed to `secrets-init`
MY_API_KEY=arn:aws:ssm:$AWS_REGION:$AWS_ACCOUNT_ID:parameter/api/key
# OR versioned parameter
MY_API_KEY=arn:aws:ssm:$AWS_REGION:$AWS_ACCOUNT_ID:parameter/api/key:$VERSION
# environment variable passed to child process, resolved by `secrets-init`
MY_API_KEY=key-123456789Integration with Google Secret Manager
User can put Google secret name (prefixed with gcp:secretmanager:) as environment variable value. The secrets-init will resolve any environment value, using specified name, to referenced secret value.
# environment variable passed to `secrets-init`
MY_DB_PASSWORD=gcp:secretmanager:projects/$PROJECT_ID/secrets/mydbpassword
# OR versioned secret (with version or 'latest')
MY_DB_PASSWORD=gcp:secretmanager:projects/$PROJECT_ID/secrets/mydbpassword/versions/2
# environment variable passed to child process, resolved by `secrets-init`
MY_DB_PASSWORD=very-secret-passwordRequirement
Container
If you are building a Docker container, make sure to include the ca-certificates package, or use already prepared doitintl/secrets-init Docker container (linux/amd64, linux/arm64).
AWS
In order to resolve AWS secrets from AWS Secrets Manager and Parameter Store, secrets-init should run under IAM role that has permission to access desired secrets.
This can be achieved by assigning IAM Role to Kubernetes Pod or ECS Task. It's possible to assign IAM Role to EC2 instance, where container is running, but this option is less secure.
Google Cloud
In order to resolve Google secrets from Google Secret Manager, secrets-init should run under IAM role that has permission to access desired secrets.
This can be achieved by assigning IAM Role to Kubernetes Pod with Workload Identity. It's possible to assign IAM Role to GCE instance, where container is running, but this option is less secure.
Kubernetes secrets-init admission webhook
The kube-secrets-init implements Kubernetes admission webhook that injects secrets-init initContainer into any Pod that references cloud secrets (AWS Secrets Manager, AWS SSM Parameter Store and Google Secrets Manager) implicitly or explicitly.
Code Reference
Initial init system code was copied from go-init project.