• Stars
    star
    286
  • Rank 144,690 (Top 3 %)
  • Language
    Python
  • License
    MIT License
  • Created almost 7 years ago
  • Updated about 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Utilities for Amazon Elastic File System (EFS)

CircleCI

efs-utils

Utilities for Amazon Elastic File System (EFS)

The efs-utils package has been verified against the following Linux distributions:

Distribution Package Type init System
Amazon Linux 2017.09 rpm upstart
Amazon Linux 2 rpm systemd
Amazon Linux 2023 rpm systemd
CentOS 7 rpm systemd
CentOS 8 rpm systemd
RHEL 7 rpm systemd
RHEL 8 rpm systemd
Fedora 28 rpm systemd
Fedora 29 rpm systemd
Fedora 30 rpm systemd
Fedora 31 rpm systemd
Fedora 32 rpm systemd
Debian 9 deb systemd
Debian 10 deb systemd
Ubuntu 16.04 deb systemd
Ubuntu 18.04 deb systemd
Ubuntu 20.04 deb systemd
OpenSUSE Leap rpm systemd
OpenSUSE Tumbleweed rpm systemd
Oracle8 rpm systemd
SLES 12 rpm systemd
SLES 15 rpm systemd

The efs-utils package has been verified against the following MacOS distributions:

Distribution init System
MacOS Big Sur launchd
MacOS Monterey launchd
MacOS Ventura launchd

README contents

Prerequisites

  • nfs-utils (RHEL/CentOS/Amazon Linux/Fedora) or nfs-common (Debian/Ubuntu)
  • OpenSSL 1.0.2+
  • Python 3.4+
  • stunnel 4.56+

Optional

  • botocore 1.12.0+

Installation

On Amazon Linux distributions

For those using Amazon Linux or Amazon Linux 2, the easiest way to install efs-utils is from Amazon's repositories:

$ sudo yum -y install amazon-efs-utils

Install via AWS Systems Manager Distributor

You can now use AWS Systems Manage Distributor to automatically install or update amazon-efs-utils. Please refer to Using AWS Systems Manager to automatically install or update Amazon EFS clients for more guidance.

The following are prerequisites for using AWS Systems Manager Distributor to install or update amazon-efs-utils:

  1. AWS Systems Manager agent is installed on the distribution (For Amazon Linux and Ubuntu, AWS Systems Manager agent is pre-installed, for other distributions, please refer to install AWS Systems Manager agent on Linux EC2 instance for more guidance.)

  2. Instance is attached with IAM role with AWS managed policy AmazonElasticFileSystemsUtils, this policy will enable your instance to be managed by AWS Systems Manager agent, also it contains permissions to support specific features.

On other Linux distributions

Other distributions require building the package from source and installing it.

  • To build and install an RPM:

If the distribution is not OpenSUSE or SLES

$ sudo yum -y install git rpm-build make
$ git clone https://github.com/aws/efs-utils
$ cd efs-utils
$ make rpm
$ sudo yum -y install build/amazon-efs-utils*rpm

Otherwise

$ sudo zypper refresh
$ sudo zypper install -y git rpm-build make
$ git clone https://github.com/aws/efs-utils
$ cd efs-utils
$ make rpm
$ sudo zypper --no-gpg-checks install -y build/amazon-efs-utils*rpm

On OpenSUSE, if you see error like File './suse/noarch/bash-completion-2.11-2.1.noarch.rpm' not found on medium 'http://download.opensuse.org/tumbleweed/repo/oss/' during installation of git, run the following commands to re-add repo OSS and NON-OSS, then run the install script above again.

sudo zypper ar -f -n OSS http://download.opensuse.org/tumbleweed/repo/oss/ OSS
sudo zypper ar -f -n NON-OSS http://download.opensuse.org/tumbleweed/repo/non-oss/ NON-OSS
sudo zypper refresh
  • To build and install a Debian package:
$ sudo apt-get update
$ sudo apt-get -y install git binutils
$ git clone https://github.com/aws/efs-utils
$ cd efs-utils
$ ./build-deb.sh
$ sudo apt-get -y install ./build/amazon-efs-utils*deb

On MacOS Big Sur, macOS Monterey and macOS Ventura distribution

For EC2 Mac instances running macOS Big Sur, macOS Monterey and macOS Ventura, you can install amazon-efs-utils from the homebrew-aws respository. Note that this will ONLY work on EC2 instances running macOS Big Sur, macOS Monterey and macOS Ventura, not local Mac computers.

brew install amazon-efs-utils

This will install amazon-efs-utils on your EC2 Mac Instance running macOS Big Sur, macOS Monterey and macOS Ventura in the directory /usr/local/Cellar/amazon-efs-utils.

Follow the instructions in caveats when using efs-utils on EC2 Mac instance for the first time. To check the package caveats run below command

brew info amazon-efs-utils

Run tests

$ virtualenv ~/.envs/efs-utils
$ source ~/.envs/efs-utils/bin/activate
$ pip install -r requirements.txt
  • Run tests
$ make test

Usage

mount.efs

efs-utils includes a mount helper utility to simplify mounting and using EFS file systems.

To mount with the recommended default options, simply run:

$ sudo mount -t efs file-system-id efs-mount-point/

To mount file system to a specific mount target of the file system, run:

$ sudo mount -t efs -o mounttargetip=mount-target-ip-address file-system-id efs-mount-point/

To mount file system within a given network namespace, run:

$ sudo mount -t efs -o netns=netns-path file-system-id efs-mount-point/

To mount file system to the mount target in specific availability zone (e.g. us-east-1a), run:

$ sudo mount -t efs -o az=az-name file-system-id efs-mount-point/

To mount over TLS, simply add the tls option:

$ sudo mount -t efs -o tls file-system-id efs-mount-point/

To authenticate with EFS using the systemโ€™s IAM identity, add the iam option. This option requires the tls option.

$ sudo mount -t efs -o tls,iam file-system-id efs-mount-point/

To mount using an access point, use the accesspoint= option. This option requires the tls option. The access point must be in the "available" state before it can be used to mount EFS.

$ sudo mount -t efs -o tls,accesspoint=access-point-id file-system-id efs-mount-point/

To mount your file system automatically with any of the options above, you can add entries to /efs/fstab like:

file-system-id efs-mount-point efs _netdev,tls,iam,accesspoint=access-point-id 0 0

For more information on mounting with the mount helper, see the manual page:

man mount.efs

or refer to the documentation.

MacOS

For EC2 instances using Mac distribution, the recommended default options will perform a tls mount:

$ sudo mount -t efs file-system-id efs-mount-point/

or

$ sudo mount -t efs -o tls file-system-id efs-mount-point/

To mount without TLS, simply add the notls option:

$ sudo mount -t efs -o notls file-system-id efs-mount-point/

amazon-efs-mount-watchdog

efs-utils contains a watchdog process to monitor the health of TLS mounts. This process is managed by either upstart or systemd depending on your Linux distribution and launchd on Mac distribution, and is started automatically the first time an EFS file system is mounted over TLS.

Troubleshooting

If you run into a problem with efs-utils, please open an issue in this repository. We can more easily assist you if relevant logs are provided. You can find the log file at /var/log/amazon/efs/mount.log.

Often times, enabling debug level logging can help us find problems more easily. To do this, run
sed -i '/logging_level = INFO/s//logging_level = DEBUG/g' /etc/amazon/efs/efs-utils.conf.

You can also enable stunnel debug logs with
sed -i '/stunnel_debug_enabled = false/s//stunnel_debug_enabled = true/g' /etc/amazon/efs/efs-utils.conf.

Make sure to perform the failed mount again after running the prior commands before pulling the logs.

Upgrading stunnel for RHEL/CentOS

By default, when using the EFS mount helper with TLS, it enforces certificate hostname checking. The EFS mount helper uses the stunnel program for its TLS functionality. Please note that some versions of Linux do not include a version of stunnel that supports TLS features by default. When using such a Linux version, mounting an EFS file system using TLS will fail.

Once youโ€™ve installed the amazon-efs-utils package, to upgrade your systemโ€™s version of stunnel, see Upgrading Stunnel.

Upgrading stunnel for SLES12

Run the following commands and follow the output hint of zypper package manager to upgrade the stunnel on your SLES12 instance

sudo zypper addrepo https://download.opensuse.org/repositories/security:Stunnel/SLE_12_SP5/security:Stunnel.repo
sudo zypper refresh
sudo zypper install -y stunnel

Upgrading stunnel for MacOS

The installation installs latest stunnel available in brew repository. You can also upgrade the version of stunnel on your instance using the command below:

brew upgrade stunnel

Install botocore

efs-utils uses botocore to interact with other AWS services. Please note the package type from the above table and install botocore based on that info. If botocore is already installed and does not meet the minimum required version, you can upgrade the botocore by following the upgrade botocore section.

  • Download the get-pip.py script

RPM

sudo yum -y install wget
if [[ "$(python3 -V 2>&1)" =~ ^(Python 3.6.*) ]]; then
    sudo wget https://bootstrap.pypa.io/pip/3.6/get-pip.py -O /tmp/get-pip.py
elif [[ "$(python3 -V 2>&1)" =~ ^(Python 3.5.*) ]]; then
    sudo wget https://bootstrap.pypa.io/pip/3.5/get-pip.py -O /tmp/get-pip.py
elif [[ "$(python3 -V 2>&1)" =~ ^(Python 3.4.*) ]]; then
    sudo wget https://bootstrap.pypa.io/pip/3.4/get-pip.py -O /tmp/get-pip.py
else
    sudo wget https://bootstrap.pypa.io/get-pip.py -O /tmp/get-pip.py
fi

DEB

sudo apt-get update
sudo apt-get -y install wget
if echo $(python3 -V 2>&1) | grep -e "Python 3.6"; then
    sudo wget https://bootstrap.pypa.io/pip/3.6/get-pip.py -O /tmp/get-pip.py
elif echo $(python3 -V 2>&1) | grep -e "Python 3.5"; then
    sudo wget https://bootstrap.pypa.io/pip/3.5/get-pip.py -O /tmp/get-pip.py
elif echo $(python3 -V 2>&1) | grep -e "Python 3.4"; then
    sudo wget https://bootstrap.pypa.io/pip/3.4/get-pip.py -O /tmp/get-pip.py
else
    sudo apt-get -y install python3-distutils
    sudo wget https://bootstrap.pypa.io/get-pip.py -O /tmp/get-pip.py
fi
  • To install botocore on RPM
sudo python3 /tmp/get-pip.py
sudo pip3 install botocore || sudo /usr/local/bin/pip3 install botocore
  • To install botocore on DEB
sudo python3 /tmp/get-pip.py
sudo pip3 install botocore || sudo /usr/local/bin/pip3 install botocore

On Debian10 and Ubuntu20, the botocore needs to be installed in specific target folder

sudo python3 /tmp/get-pip.py
sudo pip3 install --target /usr/lib/python3/dist-packages botocore || sudo /usr/local/bin/pip3 install --target /usr/lib/python3/dist-packages botocore

To install botocore on MacOS

sudo pip3 install botocore

Upgrade botocore

Pass --upgrade to the corresponding installation scripts above based on system platform and distribution

sudo pip3 install botocore --upgrade

Enable mount success/failure notification via CloudWatch log

efs-utils now support publishing mount success/failure logs to CloudWatch log. By default, this feature is disabled. There are three steps you must follow to enable and use this feature:

Step 1. Install botocore

Follow install botocore section

Step 2. Enable CloudWatch log feature in efs-utils config file /etc/amazon/efs/efs-utils.conf

sudo sed -i -e '/\[cloudwatch-log\]/{N;s/# enabled = true/enabled = true/}' /etc/amazon/efs/efs-utils.conf
  • For MacOS:
    EFS_UTILS_VERSION=<e.g. 1.34.5>
    sudo sed -i -e '/\[cloudwatch-log\]/{N;s/# enabled = true/enabled = true/;}' /usr/local/Cellar/amazon-efs-utils/${EFS_UTILS_VERSION}/libexec/etc/amazon/efs/efs-utils.conf
  • For Mac2 instance:
    EFS_UTILS_VERSION=<e.g. 1.34.5>
    sudo sed -i -e '/\[cloudwatch-log\]/{N;s/# enabled = true/enabled = true/;}' /opt/homebrew/Cellar/amazon-efs-utils/${EFS_UTILS_VERSION}/libexec/etc/amazon/efs/efs-utils.conf

You can also configure CloudWatch log group name and log retention days in the config file. If you want to have separate log groups in Cloudwatch for every mounted file system, add /{fs_id} to the end of the log_group_name field in efs-utils.conf file. For example, the log_group_name in efs-utils.conf file would look something like:

[cloudwatch-log]
log_group_name = /aws/efs/utils/{fs_id}

Step 3. Attach the CloudWatch logs policy to the IAM role attached to instance.

Attach AWS managed policy AmazonElasticFileSystemsUtils to the iam role you attached to the instance, or the aws credentials configured on your instance.

After completing the three prerequisite steps, you will be able to see mount status notifications in CloudWatch Logs.

Optimize readahead max window size on Linux 5.4+

A change in the Linux kernel 5.4+ results a throughput regression on NFS client. With patch, starting from 5.4.*, Kernels containing this patch now set the default read_ahead_kb size to 128 KB instead of the previous 15 MB. This read_ahead_kb is used by the Linux kernel to optimize performance on NFS read requests by defining the maximum amount of data an NFS client can pre-fetch in a read call. With the reduced value, an NFS client has to make more read calls to the file system, resulting in reduced performance.

To avoid above throughput regression, efs-utils will modify read_ahead_kb to 15 * rsize (could be configured via mount option, 1MB by default) after mount success on Linux 5.4+. (not support on MacOS)

This optimization will be enabled by default. To disable this optimization:

sed -i "s/optimize_readahead = false/optimize_readahead = true/" /etc/amazon/efs/efs-utils.conf

To re-enable this optimization

sed -i "s/optimize_readahead = true/optimize_readahead = false/" /etc/amazon/efs/efs-utils.conf

You can mount file system with a given rsize, run:

$ sudo mount -t efs -o rsize=rsize-value-in-bytes file-system-id efs-mount-point/

You can also manually chose a value of read_ahead_kb to optimize read throughput on Linux 5.4+ after mount.

$ sudo bash -c "echo read-ahead-value-in-kb > /sys/class/bdi/0:$(stat -c '%d' efs-mount-point)/read_ahead_kb"

Using botocore to retrieve mount target ip address when dns name cannot be resolved

efs-utils now supports using botocore to retrieve mount target ip address when dns name cannot be resolved, e.g. when user is mounting a file system in another VPC. There are two prerequisites to use this feature:

Step 1. Install botocore

Follow install botocore section

Step 2. Allow DescribeMountTargets and DescribeAvailabilityZones action in the IAM policy

Allow the elasticfilesystem:DescribeMountTargets and ec2:DescribeAvailabilityZones action in your policy attached to the iam role you attached to the instance, or the aws credentials configured on your instance. We recommend you attach AWS managed policy AmazonElasticFileSystemsUtils.

This feature will be enabled by default. To disable this feature:

sed -i "s/fall_back_to_mount_target_ip_address_enabled = true/fall_back_to_mount_target_ip_address_enabled = false/" /etc/amazon/efs/efs-utils.conf

If you decide that you do not want to use this feature, but need to mount a cross-VPC file system, you can use the mounttargetip option to do so, using the desired mount target ip address in the mount command.

The way to access instance metadata

efs-utils by default uses IMDSv2, which is a session-oriented method used to access instance metadata. If you don't want to use IMDSv2, you can disable the token fetching feature by running the following command:

sed -i "s/disable_fetch_ec2_metadata_token = false/disable_fetch_ec2_metadata_token = true/" /etc/amazon/efs/efs-utils.conf

Use the assumed profile credentials for IAM

To authenticate with EFS using the systemโ€™s IAM identity of an awsprofile, add the iam option and pass the profile name to awsprofile option. These options require the tls option.

$ sudo mount -t efs -o tls,iam,awsprofile=test-profile file-system-id efs-mount-point/

To configure the named profile, see the Named Profiles doc and Support Config File Settings doc for more details. If the credentials (e.g. aws_access_key_id) are not configured in /root/.aws/credentials or /root/.aws/config (note that the path prefix may vary based on the root path of sudo), efs-utils will use botocore to assume the named profile. This will require botocore is pre-installed, please follow install botocore section to install botocore first.

Normally you will need to configure your profile IAM policy to make the assume works. For example, if you want to perform a cross-account mounting, suppose you have established vpc-peering-connections between your vpcs, next step you need to do is giving permission to account B so that it can assume a role in account A and then mount the file system that belongs to account A. You can see IAM doc for more details.

After the IAM identity is setup, you can configure your awsprofile credentials or config. You can refer to sdk settings. For example you can define the profile to use the credentials of profile default to assume role in account A by defining the source_profile.

# /root/.aws/credentials
[default]
aws_access_key_id = AKIAIOSFODNN7EXAMPLE
aws_secret_access_key_id =wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# /root/.aws/config
[default]
...

[profile test-profile]
role_arn = <role-arn-in-account-A>
source_profile = default

Or you can use the credentials from IAM role attached to instance to assume the named profile, e.g.

# /root/.aws/config
[profile test-profile]
role_arn = <role-arn-in-account-A>
credential_source = Ec2InstanceMetadata

Use AssumeRoleWithWebIdentity

You can use web identity to assume a role which has the permission to attach to the EFS filesystem. You need to have a valid JWT token and a role arn to assume. There are two ways you can leverage them:

  1. By setting environment variable the path to the file containing the JWT token in AWS_WEB_IDENTITY_TOKEN_FILE and by setting ROLE_ARN environment variable. The command below shows an example of to leverage it.
$ sudo mount -t efs -o tls,iam file-system-id efs-mount-point/
  1. By passing the JWT token file path and the role arn as parameters to the mount command. The command below shows an example of to leverage it.
$ sudo mount -t efs -o tls,iam,rolearn="ROLE_ARN",jwtpath="PATH/JWT_TOKEN_FILE" file-system-id efs-mount-point/

Enabling FIPS Mode

Efs-Utils is able to enter FIPS mode when mounting your file system. To enable FIPS you need to modify the EFS-Utils config file:

sed -i "s/fips_mode_enabled = false/fips_mode_enabled = true/" /etc/amazon/efs/efs-utils.conf

This will enable any potential API call from EFS-Utils to use FIPS endpoints and cause stunnel to enter FIPS mode

Note: FIPS mode requires that the installed version of OpenSSL is compiled with FIPS.

To verify that the installed version is compiled with FIPS, look for OpenSSL X.X.Xx-fips in the stunnel -version command output e.g.

$ stunnel -version
stunnel 4.56 on x86_64-koji-linux-gnu platform
Compiled/running with OpenSSL 1.0.2k-fips  26 Jan 2017
Threading:PTHREAD Sockets:POLL,IPv6 SSL:ENGINE,OCSP,FIPS Auth:LIBWRAP

For more information on how to configure OpenSSL with FIPS see the OpenSSL FIPS README.

Disabling Version Check

By default, once an hour, the watchdog daemon service will check to see if a newer version of amazon-efs-utils is available on github or yum. You can disable this check by setting the enable_version_check field in /etc/amazon/efs/efs-utils.conf to false. For example,

sudo sed -i 's/enable_version_check = true/enable_version_check = false/' /etc/amazon/efs/efs-utils.conf

Or on MacOS:

VERSION=<efs-utils version, e.g. 1.34.1>
sudo sed -i 's/enable_version_check = true/enable_version_check = false/' /usr/local/Cellar/amazon-efs-utils/${VERSION}/libexec/etc/amazon/efs/efs-utils.conf

License Summary

This code is made available under the MIT license.

More Repositories

1

aws-cli

Universal Command Line Interface for Amazon Web Services
Python
14,304
star
2

chalice

Python Serverless Microframework for AWS
Python
10,654
star
3

aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
JavaScript
10,440
star
4

amazon-sagemaker-examples

Example ๐Ÿ““ Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using ๐Ÿง  Amazon SageMaker.
Jupyter Notebook
9,542
star
5

serverless-application-model

The AWS Serverless Application Model (AWS SAM) transform is a AWS CloudFormation macro that transforms SAM templates into CloudFormation templates.
Python
9,342
star
6

aws-sdk-js

AWS SDK for JavaScript in the browser and Node.js
JavaScript
7,476
star
7

aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
Python
6,506
star
8

aws-sdk-php

Official repository of the AWS SDK for PHP (@awsforphp)
PHP
5,886
star
9

containers-roadmap

This is the public roadmap for AWS container services (ECS, ECR, Fargate, and EKS).
Shell
5,164
star
10

karpenter

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
Go
4,615
star
11

s2n-tls

An implementation of the TLS/SSL protocols
C
4,465
star
12

aws-sdk-java

The official AWS SDK for Java 1.x. The AWS SDK for Java 2.x is available here: https://github.com/aws/aws-sdk-java-v2/
Java
4,117
star
13

aws-lambda-go

Libraries, samples and tools to help Go developers develop AWS Lambda functions.
Go
3,624
star
14

aws-sdk-pandas

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Python
3,537
star
15

copilot-cli

The AWS Copilot CLI is a tool for developers to build, release and operate production ready containerized applications on AWS App Runner or Amazon ECS on AWS Fargate.
Go
3,488
star
16

aws-sdk-ruby

The official AWS SDK for Ruby.
Ruby
3,462
star
17

amazon-freertos

DEPRECATED - See README.md
C
2,535
star
18

aws-sdk-go-v2

AWS SDK for the Go programming language.
Go
2,518
star
19

aws-sdk-js-v3

Modularized AWS SDK for JavaScript.
TypeScript
2,476
star
20

jsii

jsii allows code in any language to naturally interact with JavaScript classes. It is the technology that enables the AWS Cloud Development Kit to deliver polyglot libraries from a single codebase!
TypeScript
2,371
star
21

sagemaker-python-sdk

A library for training and deploying machine learning models on Amazon SageMaker
Python
2,095
star
22

amazon-vpc-cni-k8s

Networking plugin repository for pod networking in Kubernetes using Elastic Network Interfaces on AWS
Go
2,071
star
23

aws-eks-best-practices

A best practices guide for day 2 operations, including operational excellence, security, reliability, performance efficiency, and cost optimization.
Python
2,022
star
24

amazon-ecs-agent

Amazon Elastic Container Service Agent
Go
2,005
star
25

lumberyard

Amazon Lumberyard is a free AAA game engine deeply integrated with AWS and Twitch โ€“ with full source.
C++
1,965
star
26

aws-sdk-net

The official AWS SDK for .NET. For more information on the AWS SDK for .NET, see our web site:
1,945
star
27

eks-anywhere

Run Amazon EKS on your own infrastructure ๐Ÿš€
Go
1,899
star
28

aws-sdk-java-v2

The official AWS SDK for Java - Version 2
Java
1,822
star
29

aws-sdk-cpp

AWS SDK for C++
1,779
star
30

amazon-ecs-cli

The Amazon ECS CLI enables users to run their applications on ECS/Fargate using the Docker Compose file format, quickly provision resources, push/pull images in ECR, and monitor running applications on ECS/Fargate.
Go
1,725
star
31

aws-sdk-php-laravel

A Laravel 5+ (and 4) service provider for the AWS SDK for PHP
PHP
1,589
star
32

serverless-java-container

A Java wrapper to run Spring, Spring Boot, Jersey, and other apps inside AWS Lambda.
Java
1,483
star
33

aws-node-termination-handler

Gracefully handle EC2 instance shutdown within Kubernetes
Go
1,443
star
34

aws-lambda-dotnet

Libraries, samples and tools to help .NET Core developers develop AWS Lambda functions.
C#
1,430
star
35

aws-fpga

Official repository of the AWS EC2 FPGA Hardware and Software Development Kit
VHDL
1,380
star
36

eks-distro

Amazon EKS Distro (EKS-D) is a Kubernetes distribution based on and used by Amazon Elastic Kubernetes Service (EKS) to create reliable and secure Kubernetes clusters.
Shell
1,263
star
37

eks-charts

Amazon EKS Helm chart repository
Mustache
1,184
star
38

s2n-quic

An implementation of the IETF QUIC protocol
Rust
1,152
star
39

aws-toolkit-vscode

CodeWhisperer, CodeCatalyst, Local Lambda debug, SAM/CFN syntax, ECS Terminal, AWS resources
TypeScript
1,150
star
40

opsworks-cookbooks

Chef Cookbooks for the AWS OpsWorks Service
Ruby
1,058
star
41

aws-codebuild-docker-images

Official AWS CodeBuild repository for managed Docker images http://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref.html
Dockerfile
1,032
star
42

amazon-ssm-agent

An agent to enable remote management of your EC2 instances, on-premises servers, or virtual machines (VMs).
Go
975
star
43

aws-iot-device-sdk-js

SDK for connecting to AWS IoT from a device using JavaScript/Node.js
JavaScript
957
star
44

aws-iot-device-sdk-embedded-C

SDK for connecting to AWS IoT from a device using embedded C.
C
926
star
45

aws-health-tools

The samples provided in AWS Health Tools can help users to build automation and customized alerting in response to AWS Health events.
Python
887
star
46

aws-graviton-getting-started

Helping developers to use AWS Graviton2, Graviton3, and Graviton4 processors which power the 6th, 7th, and 8th generation of Amazon EC2 instances (C6g[d], M6g[d], R6g[d], T4g, X2gd, C6gn, I4g, Im4gn, Is4gen, G5g, C7g[d][n], M7g[d], R7g[d], R8g).
Python
850
star
47

aws-app-mesh-examples

AWS App Mesh is a service mesh that you can use with your microservices to manage service to service communication.
Shell
844
star
48

deep-learning-containers

AWS Deep Learning Containers (DLCs) are a set of Docker images for training and serving models in TensorFlow, TensorFlow 2, PyTorch, and MXNet.
Python
800
star
49

aws-parallelcluster

AWS ParallelCluster is an AWS supported Open Source cluster management tool to deploy and manage HPC clusters in the AWS cloud.
Python
782
star
50

aws-lambda-runtime-interface-emulator

Go
771
star
51

aws-toolkit-jetbrains

AWS Toolkit for JetBrains - a plugin for interacting with AWS from JetBrains IDEs
Kotlin
735
star
52

graph-notebook

Library extending Jupyter notebooks to integrate with Apache TinkerPop, openCypher, and RDF SPARQL.
Jupyter Notebook
706
star
53

aws-iot-device-sdk-python

SDK for connecting to AWS IoT from a device using Python.
Python
670
star
54

amazon-chime-sdk-js

A JavaScript client library for integrating multi-party communications powered by the Amazon Chime service.
TypeScript
655
star
55

amazon-ec2-instance-selector

A CLI tool and go library which recommends instance types based on resource criteria like vcpus and memory
Go
642
star
56

studio-lab-examples

Example notebooks for working with SageMaker Studio Lab. Sign up for an account at the link below!
Jupyter Notebook
625
star
57

aws-secretsmanager-agent

The AWS Secrets Manager Agent is a local HTTP service that you can install and use in your compute environments to read secrets from Secrets Manager and cache them in memory.
Rust
584
star
58

event-ruler

Event Ruler is a Java library that allows matching many thousands of Events per second to any number of expressive and sophisticated rules.
Java
564
star
59

aws-sdk-rails

Official repository for the aws-sdk-rails gem, which integrates the AWS SDK for Ruby with Ruby on Rails.
Ruby
554
star
60

aws-mwaa-local-runner

This repository provides a command line interface (CLI) utility that replicates an Amazon Managed Workflows for Apache Airflow (MWAA) environment locally.
Shell
553
star
61

amazon-eks-pod-identity-webhook

Amazon EKS Pod Identity Webhook
Go
534
star
62

aws-lambda-java-libs

Official mirror for interface definitions and helper classes for Java code running on the AWS Lambda platform.
C++
518
star
63

aws-lambda-base-images

506
star
64

aws-appsync-community

The AWS AppSync community
HTML
495
star
65

sagemaker-training-toolkit

Train machine learning models within a ๐Ÿณ Docker container using ๐Ÿง  Amazon SageMaker.
Python
493
star
66

dotnet

GitHub home for .NET development on AWS
487
star
67

aws-cdk-rfcs

RFCs for the AWS CDK
JavaScript
476
star
68

aws-sam-cli-app-templates

Python
472
star
69

aws-elastic-beanstalk-cli-setup

Simplified EB CLI installation mechanism.
Python
453
star
70

amazon-cloudwatch-agent

CloudWatch Agent enables you to collect and export host-level metrics and logs on instances running Linux or Windows server.
Go
403
star
71

secrets-store-csi-driver-provider-aws

The AWS provider for the Secrets Store CSI Driver allows you to fetch secrets from AWS Secrets Manager and AWS Systems Manager Parameter Store, and mount them into Kubernetes pods.
Go
393
star
72

amazon-braket-examples

Example notebooks that show how to apply quantum computing in Amazon Braket.
Python
376
star
73

aws-for-fluent-bit

The source of the amazon/aws-for-fluent-bit container image
Shell
375
star
74

aws-pdk

The AWS PDK provides building blocks for common patterns together with development tools to manage and build your projects.
TypeScript
361
star
75

aws-extensions-for-dotnet-cli

Extensions to the dotnet CLI to simplify the process of building and publishing .NET Core applications to AWS services
C#
346
star
76

aws-sdk-php-symfony

PHP
346
star
77

aws-app-mesh-roadmap

AWS App Mesh is a service mesh that you can use with your microservices to manage service to service communication
344
star
78

aws-lambda-builders

Python library to compile, build & package AWS Lambda functions for several runtimes & framework
Python
337
star
79

aws-iot-device-sdk-python-v2

Next generation AWS IoT Client SDK for Python using the AWS Common Runtime
Python
335
star
80

constructs

Define composable configuration models through code
TypeScript
332
star
81

pg_tle

Framework for building trusted language extensions for PostgreSQL
C
329
star
82

graph-explorer

React-based web application that enables users to visualize both property graph and RDF data and explore connections between data without having to write graph queries.
TypeScript
321
star
83

aws-codedeploy-agent

Host Agent for AWS CodeDeploy
Ruby
316
star
84

aws-sdk-ruby-record

Official repository for the aws-record gem, an abstraction for Amazon DynamoDB.
Ruby
313
star
85

aws-ops-wheel

The AWS Ops Wheel is a randomizer that biases for options that havenโ€™t come up recently; you can also outright cheat and specify the next result to be generated.
JavaScript
308
star
86

aws-xray-sdk-python

AWS X-Ray SDK for the Python programming language
Python
304
star
87

sagemaker-inference-toolkit

Serve machine learning models within a ๐Ÿณ Docker container using ๐Ÿง  Amazon SageMaker.
Python
303
star
88

amazon-ivs-react-native-player

A React Native wrapper for the Amazon IVS iOS and Android player SDKs.
TypeScript
286
star
89

sagemaker-spark

A Spark library for Amazon SageMaker.
Scala
282
star
90

apprunner-roadmap

This is the public roadmap for AWS App Runner.
280
star
91

aws-xray-sdk-go

AWS X-Ray SDK for the Go programming language.
Go
274
star
92

aws-toolkit-eclipse

(End of life: May 31, 2023) AWS Toolkit for Eclipse
Java
273
star
93

elastic-beanstalk-roadmap

AWS Elastic Beanstalk roadmap
272
star
94

aws-logging-dotnet

.NET Libraries for integrating Amazon CloudWatch Logs with popular .NET logging libraries
C#
271
star
95

sagemaker-tensorflow-training-toolkit

Toolkit for running TensorFlow training scripts on SageMaker. Dockerfiles used for building SageMaker TensorFlow Containers are at https://github.com/aws/deep-learning-containers.
Python
270
star
96

aws-lc-rs

aws-lc-rs is a cryptographic library using AWS-LC for its cryptographic operations. The library strives to be API-compatible with the popular Rust library named ring.
Rust
263
star
97

elastic-load-balancing-tools

AWS Elastic Load Balancing Tools
Java
262
star
98

aws-step-functions-data-science-sdk-python

Step Functions Data Science SDK for building machine learning (ML) workflows and pipelines on AWS
Python
261
star
99

amazon-braket-sdk-python

A Python SDK for interacting with quantum devices on Amazon Braket
Python
254
star
100

aws-xray-sdk-node

The official AWS X-Ray SDK for Node.js.
JavaScript
248
star