tobilg/lsh tobilg/lsh

  • Stars
    star
    141
  • Rank 259,298 (Top 6 %)
  • Language
    JavaScript
  • License
    MIT License
  • Created over 5 years ago
  • Updated over 1 year ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
tobilg/lsh
github

tobilg

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Run interactive shell commands on AWS Lambda

lsh - The Lambda shell

Run interactive shell commands on AWS Lambda

asciicast

Motivation

The main motivation for this project was to have a convenient way to

  • "Poke around" the AWS Lambda environment
  • Be able to have a "minimal" shell environment on AWS, without having to use more costly EC2 instances
  • Do some experiments in the AWS Lambda environment, like running Docker containers via udocker

Installation

You can install the Lambda shell (lsh) via

$ npm i -g lambda-shell

Usage

After the npm installation has finished, just run

$ lsh

You can display the help like this:

λ help

  Commands:

    help [command...]   Provides help for a given command.
    exit                Exits application.
    shell               Switch into interactive shell mode.
    install [options]   Deploy the lsh stack in your AWS account.
    uninstall           Remove the lsh stack from your AWS account.
    version             Print version information.
    config              Print the current Lambda configuration.
    reset               Reset the current Lambda configuration to the defaults.

Using credentials

As lsh uses the AWS SDK internally, the same options to specify credentials apply (please refer to the AWS SDK documentation). E.g. if you have created a credential file with multiple profiles, and want to use a specific profile, you can use the AWS_PROFILE environment variable like this:

$ AWS_PROFILE=myprofile lsh

Install stack

Before you can use lsh, you need to install the neccessary stack by issueing the following command:

λ install

You can also specify options for the installation of the stack:

λ help install

  Usage: install [options]

  Deploy the lsh stack in your AWS account.

  Options:

    --help                                  output usage information
    -b, --bucket <bucketName>               Name of the S3 bucket.
    -r, --region <regionName>               Region to which the Lambda function shall be deployed to (default: us-east-1).
    -m, --memory <memoryMegabytes>          Amount of memory in meagabytes the Lambda function shall have available (default: 1536).
    -t, --timeout <timeoutSeconds>          Timeout in seconds of the Lambda function (default: 60).
    -e, --efs-ap-arn <efsAccessPointArn>    The ARN of the preconfigured EFS AccessPoint.
    -f, --efs-fs-arn <efsFileSystemArn>     The ARN of the preconfigured EFS FileSystem.
    -p, --path <efsMountPath>               The absolute path where the EFS file system shall be mounted (needs to have /mnt/ prefix).
    -s, --security-group <securityGroupId>  The ID of the VPC SecurityGroup to use.
    -n, --subnet <subnetId>                 The ID of the VPC Subnet to use.
    -i, --iamRoleArn <iamRoleArn>           ARN of the IAM role to be used by the Lambda function. (default: role created by lsh)

For example, to use a Lambda shell with 3GB of memory with a 900 seconds timeout in the eu-central-1 region, use

λ install -m 3076 -t 900 -r eu-central-1

Configuration updates

You can update the Lambda configuration just by running install again with different options. This will trigger a CloudFormation stack update. You can then verify the changes by using the config command.

Working with the interactive shell

Once you installed the stack, you can start working with lsh:

λ shell
 ___       ________  ___  ___     
|\  \     |\   ____\|\  \|\  \    
\ \  \    \ \  \___|\ \  \\\  \   
 \ \  \    \ \_____  \ \   __  \  
  \ \  \____\|____|\  \ \  \ \  \ 
   \ \_______\____\_\  \ \__\ \__\
    \|_______|\_________\|__|\|__|
             \|_________|         
                                  
                                  
Welcome to interactive mode.
You can now directly enter arbitrary shell commands. To exit, type `exit`.
λ $

Now, you can use the Lambda shell interactively, e.g.

λ $ env
AWS_LAMBDA_FUNCTION_VERSION=$LATEST
AWS_LAMBDA_LOG_GROUP_NAME=/aws/lambda/lsh
LAMBDA_TASK_ROOT=/var/task
LD_LIBRARY_PATH=/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib
AWS_LAMBDA_LOG_STREAM_NAME=2019/06/23/[$LATEST]0bbb2d3b763b4b92a1027dedf3cbd0e2
AWS_EXECUTION_ENV=AWS_Lambda_nodejs8.10
AWS_XRAY_DAEMON_ADDRESS=169.254.79.2:2000
AWS_LAMBDA_FUNCTION_NAME=lsh
PATH=/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin:/var/task
AWS_DEFAULT_REGION=us-east-1
PWD=/tmp
LAMBDA_RUNTIME_DIR=/var/runtime
LANG=en_US.UTF-8
NODE_PATH=/opt/nodejs/node8/node_modules:/opt/nodejs/node_modules:/var/runtime/node_modules:/var/runtime:/var/task:/var/runtime/node_modules
AWS_REGION=us-east-1
TZ=:UTC
SHLVL=1
_AWS_XRAY_DAEMON_ADDRESS=169.254.79.2
_AWS_XRAY_DAEMON_PORT=2000
_X_AMZN_TRACE_ID=Root=1-5d0fa6a8-a0cb5800d19af40014ac8000;Parent=67cfd29878f54a87;Sampled=0
AWS_XRAY_CONTEXT_MISSING=LOG_ERROR
_HANDLER=index.handler
AWS_LAMBDA_FUNCTION_MEMORY_SIZE=1536
_=/usr/bin/env

Hint

As lsh is invoking the Lambda function via request-response, it's for example not possible to edit files directly in the Lambda environment.

Show current configuration

To check the current configuration, you can use

λ config
 ✓ Memory     128mb
 ✓ Timeout    120s
 ✓ Region     us-east-1
 ✓ S3 Bucket  lsh-j03nfi7agsd

The configuration can be changed by running install again and specifying different settings via the configuration options.

Reset configuration

To reset the current configuration to the defaults, you can use

λ reset
 ✓ Reset configuration to defaults
λ config
 ✓ Memory     128mb
 ✓ Timeout    60s
 ✓ Region     us-east-1
 ✓ S3 Bucket  lsh-n45lrkvtabc

Uninstall stack

To uninstall the created stack run the following command:

λ uninstall

Examples

Use EFS

To install lsh with EFS support, please create a EFS FileSystem and AccessPoint first, as outlined in this AWS article. Additionally, you need to lookup your Subnet and SecurityGroup IDs you want to use.Then, you can use the ARNs of the FileSystem and AccessPoint to install lsh (use the appropriate values):

λ install -p /mnt/efs -e arn:aws:elasticfilesystem:us-east-1:111111111111:access-point/fsap-123456789abcdef -f arn:aws:elasticfilesystem:us-east-1:111111111111:file-system/fs-acbdef123 -n subnet-abcdef123 -s sg-abcdef123

The installation can take a few minutes in this case. Once it's completed, you can use the mounted EFS FileSystem like this:

λ shell
 ___       ________  ___  ___     
|\  \     |\   ____\|\  \|\  \    
\ \  \    \ \  \___|\ \  \\\  \   
 \ \  \    \ \_____  \ \   __  \  
  \ \  \____\|____|\  \ \  \ \  \ 
   \ \_______\____\_\  \ \__\ \__\
    \|_______|\_________\|__|\|__|
             \|_________|         
                                  
                                  
Welcome to the Lambda shell!
You can now directly enter shell commands which will be run in the Lambda environment. To exit, type `exit`.
λ $ cd /mnt/efs

λ $ pwd
/mnt/efs
λ $ echo "test" > test.txt

λ $ ls -la
total 12
drwxrwxrwx 2 1000 1000 6144 Jul 25 12:10 .
drwxr-xr-x 3 root root 4096 Jul 25 12:10 ..
-rw-rw-r-- 1 1000 1000    5 Jul 25 12:10 test.txt
λ $ cat test.txt
test
λ $ rm test.txt

λ $ ls -la
total 8
drwxrwxrwx 2 1000 1000 6144 Jul 25 12:17 .
drwxr-xr-x 3 root root 4096 Jul 25 12:10 ..
λ $

Download aws-cli

λ shell 
 ___       ________  ___  ___     
|\  \     |\   ____\|\  \|\  \    
\ \  \    \ \  \___|\ \  \\\  \   
 \ \  \    \ \_____  \ \   __  \  
  \ \  \____\|____|\  \ \  \ \  \ 
   \ \_______\____\_\  \ \__\ \__\
    \|_______|\_________\|__|\|__|
             \|_________|         
                                  
                                  
Welcome to the Lambda shell!
You can now directly enter shell commands which will be run in the Lambda environment. To exit, type `exit`.
λ $ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11.6M  100 11.6M    0     0  78.0M      0 --:--:-- --:--:-- --:--:-- 78.0M
λ $ unzip awscli-bundle.zip
Archive:  awscli-bundle.zip
  inflating: awscli-bundle/install   
  inflating: awscli-bundle/packages/argparse-1.2.1.tar.gz  
  inflating: awscli-bundle/packages/rsa-3.4.2.tar.gz  
  inflating: awscli-bundle/packages/ordereddict-1.1.tar.gz  
  inflating: awscli-bundle/packages/simplejson-3.3.0.tar.gz  
  inflating: awscli-bundle/packages/urllib3-1.25.3.tar.gz  
  inflating: awscli-bundle/packages/python-dateutil-2.6.1.tar.gz  
  inflating: awscli-bundle/packages/s3transfer-0.2.1.tar.gz  
  inflating: awscli-bundle/packages/six-1.12.0.tar.gz  
  inflating: awscli-bundle/packages/python-dateutil-2.8.0.tar.gz  
  inflating: awscli-bundle/packages/virtualenv-15.1.0.tar.gz  
  inflating: awscli-bundle/packages/jmespath-0.9.4.tar.gz  
  inflating: awscli-bundle/packages/urllib3-1.22.tar.gz  
  inflating: awscli-bundle/packages/botocore-1.12.175.tar.gz  
  inflating: awscli-bundle/packages/colorama-0.3.9.tar.gz  
  inflating: awscli-bundle/packages/PyYAML-3.13.tar.gz  
  inflating: awscli-bundle/packages/pyasn1-0.4.5.tar.gz  
  inflating: awscli-bundle/packages/docutils-0.14.tar.gz  
  inflating: awscli-bundle/packages/PyYAML-5.1.tar.gz  
  inflating: awscli-bundle/packages/futures-3.2.0.tar.gz  
  inflating: awscli-bundle/packages/awscli-1.16.185.tar.gz  
  inflating: awscli-bundle/packages/setup/setuptools_scm-1.15.7.tar.gz  
λ $ ./awscli-bundle/install -b /tmp/bin/aws
Running cmd: /usr/bin/python virtualenv.py --no-download --python /usr/bin/python /tmp/.local/lib/aws
Running cmd: /tmp/.local/lib/aws/bin/pip install --no-cache-dir --no-index --find-links file:///tmp/awscli-bundle/packages/setup setuptools_scm-1.15.7.tar.gz
Running cmd: /tmp/.local/lib/aws/bin/pip install --no-cache-dir --no-index --find-links file:///tmp/awscli-bundle/packages awscli-1.16.185.tar.gz
You can now run: /tmp/bin/aws --version
λ $ /tmp/bin/aws --version
aws-cli/1.16.185 Python/2.7.16 Linux/4.14.123-95.109.amzn2.x86_64 exec-env/AWS_Lambda_nodejs8.10 botocore/1.12.175
λ $

More Repositories

1

facebook-events-by-location

[DEPRECATED] A Express.js-based webservice to get public Facebook events by location
JavaScript
332
star
2

facebook-events-by-location-core

[DEPRECATED] Search Facebook events by location and proximity.
JavaScript
139
star
3

aws-fullstack-website

Deploy your fullstack websites without all the hassle on AWS with CloudFront, S3, ACM, Route53, API Gateway and Lambda via Serverless.
JavaScript
137
star
4

serverless-aws-static-websites

Deploy your static websites without all the hassle on AWS with CloudFront, S3, ACM and Route53 via Serverless
JavaScript
128
star
5

python-lambda-layer-builder

A build tool for creating optimized Python AWS Lambda layers
Shell
119
star
6

api2html

A CLI tool to transform Swagger/OpenAPI/AsyncAPI docs to beautiful HTML pages via Shins/Widdershins.
JavaScript
105
star
7

mesos-framework

A wrapper around the Mesos HTTP APIs for Schedulers and Executors. Write your Mesos framework in pure JavaScript!
JavaScript
61
star
8

aws-edge-locations

List of AWS edge location code prefixes
JavaScript
44
star
9

marathon-slack

Integration for Marathon's Event Bus with Slack
JavaScript
42
star
10

netlify-functions-landingpage

A serverless-less landing page project on Netlify, including a Mailchimp mailing list signup via functions.
JavaScript
35
star
11

docker-zookeeper-webui

Docker image for using `zk-web` as ZooKeeper Web UI
Shell
32
star
12

duckdb-nodejs-layer

Packaging DuckDB for Lambda functions. Example application: https://github.com/tobilg/serverless-duckdb
Python
22
star
13

docker-mongodb-marathon

[DEPRECATED] Please use https://github.com/tobilg/dcos-commons/tree/mongodb-replicaset/frameworks/mongodb-replicaset
JavaScript
18
star
14

mesos-envoy-sds

An Envoy Service Discovery Service for Mesos
JavaScript
17
star
15

serverless-cubejs

Serverless Cube.js backend infrastructure on AWS
JavaScript
17
star
16

mesos-framework-boilerplate

A boilerplate for developing Mesos frameworks with JavaScript
JavaScript
16
star
17

docker-livy

A Docker image for Livy, the REST Spark Server
Shell
15
star
18

gitlab-ci-runner-marathon

A customized Docker image for running scalable GitLab CI runners on Marathon
Shell
15
star
19

aws-lambda-api-call-recorder

A recorder of AWS API calls for Lambda functions
JavaScript
14
star
20

serverless-duckdb

An example of how to run DuckDB on AWS Lambda & API Gateway.
JavaScript
13
star
21

docker-predictionio

Docker container for the latest prediction.io version with most recent dependencies
Shell
12
star
22

mesos-js-framework

[DEPRECATED] Use https://github.com/tobilg/mesos-framework instead
JavaScript
8
star
23

docker-mesos-dns

A lightweight Mesos DNS Docker image. Currently based on v0.5.2
Shell
7
star
24

serverless-parquet-repartitioner

Lambda function to serverlessly repartition parquet files in S3
JavaScript
7
star
25

cfn-cur-report-definition

A custom CloudFormation resource for Cost and Usage Report ReportDefinitions
JavaScript
7
star
26

mesosdns-resolver

A bash script to resolve Mesos DNS SRV records to actual host:port endpoints
Shell
7
star
27

tile38-lambda-layer

A AWS Lambda layer for Tile38
Shell
6
star
28

docker-kafka-marathon

[DEPRECATED] A Docker image of Apache Kafka to be run on a Mesos cluster via Marathon
Shell
6
star
29

marathon-event-bus-client

A Node.js package for listening to the Marathon Event Bus
JavaScript
6
star
30

ceph-admin-ops-client

A Node.js wrapper for Ceph Object Gateway's Admin Ops API
JavaScript
6
star
31

nodejs-lambda-layer-builder

Build custom AWS Lambda layers for Node
Shell
6
star
32

buffered-queue

A simple to use buffering queue (no dependencies) which flexibly buffers objects, strings, integers etc. until either a maximum size is reached, or an interval has come to an end.
JavaScript
6
star
33

mesos-operator-api-client

Node.js client for the Mesos Operator HTTP API events (see http://mesos.apache.org/documentation/latest/operator-http-api/)
JavaScript
6
star
34

docker-mini-webserver

Provides a server (via Express.js) to expose static files from the Docker host via HTTP
JavaScript
5
star
35

npm-stats-cli

A command line interface for npm statistics and insights
JavaScript
5
star
36

cfn-cognito-resource-server

A custom CloudFormation resource for Cognito Resource Servers
JavaScript
4
star
37

global-reverse-proxy

Building a global reverse proxy with on-demand SSL support on AWS
JavaScript
3
star
38

public-cloud-provider-ip-ranges

Unified datasets for public cloud provider IP ranges. Providers include AWS, Azure, CloudFlare, DigitalOcean, Fastly, Google Cloud and Oracle Cloud.
Shell
3
star
39

docker-spark-jobserver

A Docker container for Spark Jobserver
Shell
3
star
40

clair-dcos

A Docker image for running Clair on DC/OS
Dockerfile
2
star
41

docker-mesos-spark-shell

A docker image for creating a Spark shell on a Mesos cluster
Shell
2
star
42

docker-expose-socket

Docker image for exposing the Docker UNIX socket via TCP without having to reconfigure the daemon itself.
Shell
2
star
43

serverless-twitter-bot

An example application which tweets GitHub repo releases
JavaScript
1
star
44

GeoLocateURL

Node.js project to get geo information from an URL.
JavaScript
1
star
45

mesosdns-cli

A Node.js-based CLI for querying Mesos DNS service names
JavaScript
1
star
46

minio-dcos

Example on how to launch Minio in distributed mode (4 instances) on DC/OS
Shell
1
star
47

marathon-event-proxy

A proxy for the Server Sent Events endpoint of Marathon's Event Bus
JavaScript
1
star
48

docker-chronos

A Docker container for the latest Mesosphere Chronos version
Shell
1
star
49

nextjs-duckdb

Example project running for running DuckDB on Vercel
1
star
50

iterm2-ohmyzsh-config

Configuration for iTerm2 and oh-my-zsh
1
star