• Stars
    star
    2,722
  • Rank 15,995 (Top 0.4 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created almost 8 years ago
  • Updated 9 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Repository for BLESS, an SSH Certificate Authority that runs as a AWS Lambda function

Archived

With the existence of more SSH certificate tools since the release of BLESS, and better SSH access management from AWS, we're moving BLESS to the archived OSS project state. This means we no longer plan to maintain the project, but will be keeping it public for others who may still use it.

alt text

BLESS - Bastion's Lambda Ephemeral SSH Service

Build Status Test coverage Join the chat at https://gitter.im/Netflix/bless NetflixOSS Lifecycle

BLESS is an SSH Certificate Authority that runs as an AWS Lambda function and is used to sign SSH public keys.

SSH Certificates are an excellent way to authorize users to access a particular SSH host, as they can be restricted for a single use case, and can be short lived. Instead of managing the authorized_keys of a host, or controlling who has access to SSH Private Keys, hosts just need to be configured to trust an SSH CA.

BLESS should be run as an AWS Lambda in an isolated AWS account. Because BLESS needs access to a private key which is trusted by your hosts, an isolated AWS account helps restrict who can access that private key, or modify the BLESS code you are running.

AWS Lambda functions can use an AWS IAM Policy to limit which IAM Roles can invoke the Lambda Function. If properly configured, you can restrict which IAM Roles can request SSH Certificates. For example, your SSH Bastion (aka SSH Jump Host) can run with the only IAM Role with access to invoke a BLESS Lambda Function configured with the SSH CA key trusted by the instances accessible to that SSH Bastion.

Getting Started

These instructions are to get BLESS up and running in your local development environment.

Installation Instructions

Clone the repo:

$ git clone [email protected]:Netflix/bless.git

Cd to the bless repo:

$ cd bless

Create a virtualenv if you haven't already:

$ python3.8 -m venv venv

Activate the venv:

$ source venv/bin/activate

Install package and test dependencies:

(venv) $ make develop

Run the tests:

(venv) $ make test

Deployment

To deploy an AWS Lambda Function, you need to provide a .zip with the code and all dependencies. The .zip must contain your lambda code and configurations at the top level of the .zip. The BLESS Makefile includes a publish target to package up everything into a deploy-able .zip if they are in the expected locations. You will need to setup your own Python 3.7 lambda to deploy the .zip to.

Previously the AWS Lambda Handler needed to be set to bless_lambda.lambda_handler, and this would generate a user cert. bless_lambda.lambda_handler still works for user certs. bless_lambda_user.lambda_handler_user is a handler that can also be used to issue user certificates.

A new handler bless_lambda_host.lambda_handler_host has been created to allow for the creation of host SSH certs.

All three handlers exist in the published .zip.

Compiling BLESS Lambda Dependencies

To deploy code as a Lambda Function, you need to package up all of the dependencies. You will need to compile and include your dependencies before you can publish a working AWS Lambda.

BLESS uses a docker container running Amazon Linux 2 to package everything up:

  • Execute make lambda-deps and this will run a container and save all the dependencies in ./aws_lambda_libs

Protecting the CA Private Key

  • Generate a password protected RSA Private Key in the PEM format:
$ ssh-keygen -t rsa -b 4096 -m PEM -f bless-ca- -C "SSH CA Key"
  • Note: OpenSSH Private Key format is not supported.
  • Use KMS to encrypt your password. You will need a KMS key per region, and you will need to encrypt your password for each region. You can use the AWS Console to paste in a simple lambda function like this:
import boto3
import base64
import os


def lambda_handler(event, context):
    region = os.environ['AWS_REGION']
    client = boto3.client('kms', region_name=region)
    response = client.encrypt(
    KeyId='alias/your_kms_key',
    Plaintext='Do not forget to delete the real plain text when done'
    )

    ciphertext = response['CiphertextBlob']
    return base64.b64encode(ciphertext)
  • Manage your Private Keys .pem files and passwords outside of this repo.
  • Update your bless_deploy.cfg with your Private Key's filename and encrypted passwords.
  • Provide your desired ./lambda_configs/ca_key_name.pem prior to Publishing a new Lambda .zip
  • Set the permissions of ./lambda_configs/ca_key_name.pem to 444.

You can now provide your private key and/or encrypted private key password via the lambda environment or config file. In the [Bless CA] section, you can set ca_private_key instead of the ca_private_key_file with a base64 encoded version of your .pem (e.g. cat key.pem | base64 ).

Because every config file option is supported in the environment, you can also just set bless_ca_default_password and/or bless_ca_ca_private_key. Due to limits on AWS Lambda environment variables, you'll need to compress RSA 4096 private keys, which you can now do by setting bless_ca_ca_private_key_compression. For example, set bless_ca_ca_private_key_compression = bz2 and bless_ca_ca_private_key to the output of cat ca-key.pem | bzip2 | base64.

BLESS Config File

  • Refer to the the Example BLESS Config File and its included documentation.
  • Manage your bless_deploy.cfg files outside of this repo.
  • Provide your desired ./lambda_configs/bless_deploy.cfg prior to Publishing a new Lambda .zip
  • The required [Bless CA] option values must be set for your environment.
  • Every option can be changed in the environment. The environment variable name is constructed as section_name_option_name (all lowercase, spaces replaced with underscores).

Publish Lambda .zip

  • Provide your desired ./lambda_configs/ca_key_name.pem prior to Publishing
  • Provide your desired BLESS Config File at ./lambda_configs/bless_deploy.cfg prior to Publishing
  • Provide the compiled dependencies at ./aws_lambda_libs
  • run:
(venv) $ make publish
  • deploy ./publish/bless_lambda.zip to AWS via the AWS Console, AWS SDK, or S3
  • remember to deploy it to all regions.

Lambda Requirements

You should deploy this function into its own AWS account to limit who has access to modify the code, configs, or IAM Policies. An isolated account also limits who has access to the KMS keys used to protect the SSH CA Key.

The BLESS Lambda function should run as its own IAM Role and will need access to an AWS KMS Key in each region where the function is deployed. The BLESS IAMRole will also need permissions to obtain random from kms (kms:GenerateRandom) and permissions for logging to CloudWatch Logs (logs:CreateLogGroup,logs:CreateLogStream,logs:PutLogEvents).

Using BLESS

After you have deployed BLESS you can run the sample BLESS Client from a system with access to the required AWS Credentials. This client is really just a proof of concept to validate that you have a functional lambda being called with valid IAM credentials.

(venv) $ ./bless_client.py region lambda_function_name bastion_user bastion_user_ip remote_usernames bastion_source_ip bastion_command <id_rsa.pub to sign> <output id_rsa-cert.pub>

Verifying Certificates

You can inspect the contents of a certificate with ssh-keygen directly:

$ ssh-keygen -L -f your-cert.pub

Enabling BLESS Certificates On Servers

Add the following line to /etc/ssh/sshd_config:

TrustedUserCAKeys /etc/ssh/cas.pub

Add a new file, owned by and only writable by root, at /etc/ssh/cas.pub with the contents:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQ…  #id_rsa.pub of an SSH CA
ssh-rsa AAAAB3NzaC1yc2EAAAADAQ…  #id_rsa.pub of an offline SSH CA
ssh-rsa AAAAB3NzaC1yc2EAAAADAQ…  #id_rsa.pub of an offline SSH CA 2

To simplify SSH CA Key rotation you should provision multiple CA Keys, and leave them offline until you are ready to rotate them.

Additional information about the TrustedUserCAKeys file is here

Project resources

More Repositories

1

Hystrix

Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.
Java
23,594
star
2

chaosmonkey

Chaos Monkey is a resiliency tool that helps applications tolerate random instance failures.
Go
14,410
star
3

zuul

Zuul is a gateway service that provides dynamic routing, monitoring, resiliency, security, and more.
Java
12,993
star
4

conductor

Conductor is a microservices orchestration engine.
Java
12,920
star
5

eureka

AWS Service registry for resilient mid-tier load balancing and failover.
Java
11,991
star
6

falcor

A JavaScript library for efficient data fetching
JavaScript
10,338
star
7

pollyjs

Record, Replay, and Stub HTTP Interactions.
JavaScript
10,184
star
8

SimianArmy

Tools for keeping your cloud operating in top form. Chaos Monkey is a resiliency tool that helps applications tolerate random instance failures.
Java
7,955
star
9

metaflow

🚀 Build and manage real-life ML, AI, and data science projects with ease!
Python
7,498
star
10

fast_jsonapi

No Longer Maintained - A lightning fast JSON:API serializer for Ruby Objects.
Ruby
5,078
star
11

dispatch

All of the ad-hoc things you're doing to manage incidents today, done for you, and much more!
Python
4,548
star
12

ribbon

Ribbon is a Inter Process Communication (remote procedure calls) library with built in software load balancers. The primary usage model involves REST calls with various serialization scheme support.
Java
4,468
star
13

security_monkey

Security Monkey monitors AWS, GCP, OpenStack, and GitHub orgs for assets and their changes over time.
Python
4,349
star
14

vmaf

Perceptual video quality assessment based on multi-method fusion.
Python
4,159
star
15

dynomite

A generic dynamo implementation for different k-v storage engines
C
4,104
star
16

vizceral

WebGL visualization for displaying animated traffic graphs
JavaScript
4,047
star
17

vector

Vector is an on-host performance monitoring framework which exposes hand picked high resolution metrics to every engineer’s browser.
JavaScript
3,588
star
18

atlas

In-memory dimensional time series database.
Scala
3,331
star
19

concurrency-limits

Java
3,117
star
20

consoleme

A Central Control Plane for AWS Permissions and Access
Python
3,061
star
21

flamescope

FlameScope is a visualization tool for exploring different time ranges as Flame Graphs.
Python
2,979
star
22

dgs-framework

GraphQL for Java with Spring Boot made easy.
Kotlin
2,963
star
23

archaius

Library for configuration management API
Java
2,435
star
24

asgard

[Asgard is deprecated at Netflix. We use Spinnaker ( www.spinnaker.io ).] Web interface for application deployments and cloud management in Amazon Web Services (AWS). Binary download: http://github.com/Netflix/asgard/releases
Groovy
2,235
star
25

curator

ZooKeeper client wrapper and rich ZooKeeper framework
Java
2,138
star
26

titus

1,995
star
27

EVCache

A distributed in-memory data store for the cloud
Java
1,900
star
28

lemur

Repository for the Lemur Certificate Manager
Python
1,651
star
29

bpftop

bpftop provides a dynamic real-time view of running eBPF programs. It displays the average runtime, events per second, and estimated total CPU % for each program.
Rust
1,647
star
30

genie

Distributed Big Data Orchestration Service
Java
1,635
star
31

metacat

Java
1,555
star
32

netflix.github.com

HTML
1,419
star
33

servo

Netflix Application Monitoring Library
Java
1,408
star
34

mantis

A platform that makes it easy for developers to build realtime, cost-effective, operations-focused applications
Java
1,382
star
35

vectorflow

D
1,287
star
36

hubcommander

A Slack bot for GitHub organization management -- and other things too
Python
1,262
star
37

rend

A memcached proxy that manages data chunking and L1 / L2 caches
Go
1,174
star
38

hollow

Hollow is a java library and toolset for disseminating in-memory datasets from a single producer to many consumers for high performance read-only access.
Java
1,148
star
39

repokid

AWS Least Privilege for Distributed, High-Velocity Deployment
Python
1,084
star
40

astyanax

Cassandra Java Client
Java
1,034
star
41

Priam

Co-Process for backup/recovery, Token Management, and Centralized Configuration management for Cassandra.
Java
1,024
star
42

aminator

A tool for creating EBS AMIs. This tool currently works for CentOS/RedHat Linux images and is intended to run on an EC2 instance.
Python
938
star
43

Turbine

SSE Stream Aggregator
Java
831
star
44

governator

Governator is a library of extensions and utilities that enhance Google Guice to provide: classpath scanning and automatic binding, lifecycle management, configuration to field mapping, field validation and parallelized object warmup.
Java
821
star
45

Fido

C#
816
star
46

suro

Netflix's distributed Data Pipeline
Java
783
star
47

security-bulletins

Security Bulletins that relate to Netflix Open Source
734
star
48

spectator

Client library for collecting metrics.
Java
713
star
49

Fenzo

Extensible Scheduler for Mesos Frameworks
Java
703
star
50

msl

Message Security Layer
C++
687
star
51

unleash

Professionally publish your JavaScript modules in one keystroke
JavaScript
588
star
52

denominator

Portably control DNS clouds using java or bash
Java
573
star
53

blitz4j

Logging framework for fast asynchronous logging
Java
559
star
54

edda

AWS API Read Cache
Scala
554
star
55

PigPen

Map-Reduce for Clojure
Clojure
551
star
56

netflix-graph

Compact in-memory representation of directed graph data
Java
548
star
57

go-env

a golang library to manage environment variables
Go
542
star
58

karyon

The nucleus or the base container for Applications and Services built using the NetflixOSS ecosystem
Java
495
star
59

Prana

A sidecar for your NetflixOSS based services.
Java
492
star
60

iceberg

Iceberg is a table format for large, slow-moving tabular data
Java
465
star
61

Lipstick

Pig Visualization framework
JavaScript
464
star
62

Surus

Java
453
star
63

aws-autoscaling

Tools and Documentation about using Auto Scaling
Shell
429
star
64

go-expect

an expect-like golang library to automate control of terminal or console based programs.
Go
422
star
65

nf-data-explorer

The Data Explorer gives you fast, safe access to data stored in Cassandra, Dynomite, and Redis.
TypeScript
420
star
66

Workflowable

Ruby
370
star
67

osstracker

Github organization OSS metrics collector and metrics dashboard
Scala
365
star
68

vizceral-example

Example Vizceral app
JavaScript
363
star
69

ndbench

Netflix Data Store Benchmark
HTML
360
star
70

Raigad

Co-Process for backup/recovery, Auto Deployments and Centralized Configuration management for ElasticSearch
Java
346
star
71

recipes-rss

RSS Reader Recipes that uses several of the Netflix OSS components
Java
339
star
72

aegisthus

A Bulk Data Pipeline out of Cassandra
Java
323
star
73

titus-control-plane

Titus is the Netflix Container Management Platform that manages containers and provides integrations to the infrastructure ecosystem.
Java
316
star
74

weep

The ConsoleMe CLI utility
Go
311
star
75

metaflow-ui

🎨 UI for monitoring your Metaflow executions!
TypeScript
300
star
76

dyno-queues

Dyno Queues is a recipe that provides task queues utilizing Dynomite.
Java
264
star
77

image_compression_comparison

Image Compression Comparison Framework
Python
258
star
78

falcor-express-demo

Demonstration Falcor end point for a Netflix-style Application using express
HTML
246
star
79

gradle-template

Java
244
star
80

ember-nf-graph

Composable graphing component library for EmberJS.
JavaScript
241
star
81

falcor-router-demo

A demonstration of how to build a Router for a Netflix-like application
JavaScript
236
star
82

titus-executor

Titus Executor is the container runtime/executor implementation for Titus
Go
233
star
83

photon

Photon is a Java implementation of the Interoperable Master Format (IMF) standard. IMF is a SMPTE standard whose core constraints are defined in the specification st2067-2:2013
Java
233
star
84

dial-reference

C
228
star
85

s3mper

s3mper - Consistent Listing for S3
Java
218
star
86

ReactiveLab

Experiments and prototypes with reactive application design.
Java
209
star
87

inviso

JavaScript
205
star
88

NfWebCrypto

Web Cryptography API Polyfill
C++
205
star
89

staash

A language-agnostic as well as storage-agnostic web interface for storing data into persistent storage systems, the metadata layer abstracts a lot of storage details and the pattern automation APIs take care of automating common data access patterns.
Java
204
star
90

zeno

Netflix's In-Memory Data Propagation Framework
Java
200
star
91

brutal

A multi-network asynchronous chat bot framework using twisted
Python
200
star
92

vizceral-react

JavaScript
199
star
93

dispatch-docker

Shell
193
star
94

pytheas

Web Resources and UI Framework
JavaScript
187
star
95

dyno

Java client for Dynomite
Java
184
star
96

hal-9001

Hal-9001 is a Go library that offers a number of facilities for creating a bot and its plugins.
Go
178
star
97

metaflow-service

🚀 Metadata tracking and UI service for Metaflow!
Python
173
star
98

Nicobar

Java
171
star
99

lemur-docker

Docker files for the Lemur certificate orchestration tool
Python
170
star
100

yetch

Yet-another-fetch polyfill library. Supports AbortController/AbortSignal
JavaScript
168
star