• Stars
    star
    554
  • Rank 80,342 (Top 2 %)
  • Language
    Ruby
  • License
    Other
  • Created over 9 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Official repository for the aws-sdk-rails gem, which integrates the AWS SDK for Ruby with Ruby on Rails.

AWS SDK for Ruby Rails Plugin

Gem Version Build Status Github forks Github stars

A Ruby on Rails plugin that integrates AWS services with your application using the latest version of AWS SDK For Ruby.

Installation

Add this gem to your Rails project's Gemfile:

gem 'aws-sdk-rails'

This gem also brings in the following AWS gems:

  • aws-sdk-ses
  • aws-sdk-sesv2
  • aws-sdk-sqs
  • aws-record
  • aws-sessionstore-dynamodb

If you want to use other services (such as S3), you will still need to add them to your Gemfile:

gem 'aws-sdk-rails', '~> 3'
gem 'aws-sdk-s3', '~> 1'

You will have to ensure that you provide credentials for the SDK to use. See the latest AWS SDK for Ruby Docs for details.

If you're running your Rails application on Amazon EC2, the AWS SDK will check Amazon EC2 instance metadata for credentials to load. Learn more: IAM Roles for Amazon EC2

Features

AWS SDK uses the Rails logger

The AWS SDK is configured to use the built-in Rails logger for any SDK log output. The logger is configured to use the :info log level. You can change the log level by setting :log_level in the Aws.config hash.

Aws.config.update(log_level: :debug)

Rails 5.2+ Encrypted Credentials

If you are using Rails 5.2+ Encrypted Credentials, the credentials will be decrypted and loaded under the :aws top level key:

# config/credentials.yml.enc
# viewable with: `rails credentials:edit`
aws:
  access_key_id: YOUR_KEY_ID
  secret_access_key: YOUR_ACCESS_KEY

Encrypted Credentials will take precedence over any other AWS Credentials that may exist in your environment (eg: credentials from profiles set in ~/.aws/credentials).

If you are using ActiveStorage with S3 then you do not need to specify your credentials in your storage.yml configuration: they will be loaded automatically.

DynamoDB Session Store

You can configure session storage in Rails to use DynamoDB instead of cookies, allowing access to sessions from other applications and devices. You will need to have an existing Amazon DynamoDB session table to use this feature.

You can generate a migration file for the session table using the following command ( is optional):

rails generate dynamo_db:session_store_migration <MigrationName>

The session store migration generator command will generate two files: a migration file, db/migration/#{VERSION}_#{MIGRATION_NAME}.rb, and a configuration YAML file, config/dynamo_db_session_store.yml.

The migration file will create and delete a table with default options. These options can be changed prior to running the migration and are documented in the Table class.

To create the table, run migrations as normal with:

rails db:migrate

Next, configure the Rails session store to be :dynamodb_store by editing config/initializers/session_store.rb to contain the following:

# config/initializers/session_store.rb
Rails.application.config.session_store :dynamodb_store, key: '_your_app_session'

You can now start your Rails application with session support.

Configuration

You can configure the session store with code, YAML files, or ENV, in this order of precedence. To configure in code, you can directly pass options to your initializer like so:

# config/initializers/session_store.rb
Rails.application.config.session_store :dynamodb_store,
  key: '_your_app_session',
  table_name: 'foo',
  dynamo_db_client: my_ddb_client

Alternatively, you can use the generated YAML configuration file config/dynamo_db_session_store.yml. YAML configuration may also be specified per environment, with environment configuration having precedence. To do this, create config/dynamo_db_session_store/#{Rails.env}.yml files as needed.

For configuration options, see the Configuration class.

Rack Configuration

DynamoDB session storage is implemented in the `aws-sessionstore-dynamodb` gem. The Rack middleware inherits from the `Rack::Session::Abstract::Persisted` class, which also includes additional options (such as :key) that can be passed into the Rails initializer.

Cleaning old sessions

By default sessions do not expire. See config/dynamo_db_session_store.yml to configure the max age or stale period of a session.

You can use the DynamoDB Time to Live (TTL) feature on the expire_at attribute to automatically delete expired items.

Alternatively, a Rake task for garbage collection is provided:

rake dynamo_db:collect_garbage

Amazon Simple Email Service (SES) as an ActionMailer Delivery Method

This gem will automatically register SES and SESV2 as ActionMailer delivery methods. You simply need to configure Rails to use it in your environment configuration:

# for e.g.: config/environments/production.rb
config.action_mailer.delivery_method = :ses # or :sesv2

Override credentials or other client options

Client options can be overridden by re-registering the mailer with any set of SES or SESV2 Client options. You can create a Rails initializer config/initializers/aws.rb with contents similar to the following:

require 'json'

# Assuming a file "path/to/aws_secrets.json" with contents like:
#
#     { "AccessKeyId": "YOUR_KEY_ID", "SecretAccessKey": "YOUR_ACCESS_KEY" }
#
# Remember to exclude "path/to/aws_secrets.json" from version control, e.g. by
# adding it to .gitignore
secrets = JSON.load(File.read('path/to/aws_secrets.json'))
creds = Aws::Credentials.new(secrets['AccessKeyId'], secrets['SecretAccessKey'])

Aws::Rails.add_action_mailer_delivery_method(
  :ses, # or :sesv2
  credentials: creds,
  region: 'us-east-1',
  # some other config
)

Using ARNs with SES

This gem uses `Aws::SES::Client#send_raw_email` and `Aws::SESV2::Client#send_email` to send emails. This operation allows you to specify a cross-account identity for the email's Source, From, and Return-Path. To set these ARNs, use any of the following headers on your Mail::Message object returned by your Mailer class:

  • X-SES-SOURCE-ARN

  • X-SES-FROM-ARN

  • X-SES-RETURN-PATH-ARN

Example:

# in your Rails controller
message = MyMailer.send_email(options)
message['X-SES-FROM-ARN'] = 'arn:aws:ses:us-west-2:012345678910:identity/[email protected]'
message.deliver

Active Support Notification Instrumentation for AWS SDK calls

To add ActiveSupport::Notifications Instrumentation to all AWS SDK client operations call Aws::Rails.instrument_sdk_operations before you construct any SDK clients.

Example usage in config/initializers/instrument_aws_sdk.rb

Aws::Rails.instrument_sdk_operations

Events are published for each client operation call with the following event name: ..aws. For example, S3's put_object has an event name of: put_object.S3.aws. The service name will always match the namespace of the service client (eg Aws::S3::Client => 'S3'). The payload of the event is the request context.

You can subscribe to these events as you would other ActiveSupport::Notifications:

ActiveSupport::Notifications.subscribe('put_object.S3.aws') do |name, start, finish, id, payload|
 # process event
end

# Or use a regex to subscribe to all service notifications
ActiveSupport::Notifications.subscribe(/S3[.]aws/) do |name, start, finish, id, payload|
 # process event
end

AWS SQS Active Job

This package provides a lightweight, high performance SQS backend for ActiveJob.

To use AWS SQS ActiveJob as your queuing backend, simply set the active_job.queue_adapter to :amazon or :amazon_sqs (note, :amazon has been used for a number of other Amazon rails adapters such as ActiveStorage, so has been carried forward as convention here). For details on setting the queuing backend see: ActiveJob: Setting the Backend. To use the non-blocking (async) adapter set active_job.queue_adapter to :amazon_sqs_async. If you have a lot of jobs to queue or you need to avoid the extra latency from an SQS call in your request then consider using the async adapter. However, you may also want to configure a async_queue_error_handler to handle errors that may occur when queuing jobs. See the Aws::Rails::SqsActiveJob::Configuration for documentation.

# config/application.rb
module YourApp
  class Application < Rails::Application
    config.active_job.queue_adapter = :amazon_sqs # note: can use either :amazon or :amazon_sqs
    # To use the non-blocking async adapter:
    # config.active_job.queue_adapter = :amazon_sqs_async
  end
end

# Or to set the adapter for a single job:
class YourJob < ApplicationJob
  self.queue_adapter = :amazon_sqs
  #....
end

You also need to configure a mapping of ActiveJob queue name to SQS Queue URL. For more details, see the configuration section below.

# config/aws_sqs_active_job.yml
queues:
  default: 'https://my-queue-url.amazon.aws'

To queue a job, you can just use standard ActiveJob methods:

# To queue for immediate processing
YourJob.perform_later(args)

# or to schedule a job for a future time:
YourJob.set(wait: 1.minute).perform_later(args)

Note: Due to limitations in SQS, you cannot schedule jobs for later than 15 minutes in the future.

Retry Behavior and Handling Errors

See the Rails ActiveJob Guide on Exceptions for background on how ActiveJob handles exceptions and retries.

In general - you should configure retries for your jobs using retry_on. When configured, ActiveJob will catch the exception and reschedule the job for re-execution after the configured delay. This will delete the original message from the SQS queue and requeue a new message.

By default SQS ActiveJob is configured with retry_standard_error set to true and will not delete messages for jobs that raise a StandardError and that do not handle that error via retry_on or discard_on. These job messages will remain on the queue and will be re-read and retried following the SQS Queue's configured retry and DLQ settings. If you do not have a DLQ configured, the message will continue to be attempted until it reaches the queues retention period. In general, it is a best practice to configure a DLQ to store unprocessable jobs for troubleshooting and redrive.

If you want failed jobs that do not have retry_on or discard_on configured to be immediately discarded and not left on the queue, set retry_standard_error to false. See the configuration section below for details.

Running workers - polling for jobs

To start processing jobs, you need to start a separate process (in additional to your Rails app) with bin/aws_sqs_active_job (an executable script provided with this gem). You need to specify the queue to process jobs from:

RAILS_ENV=development bundle exec aws_sqs_active_job --queue default

To see a complete list of arguments use --help.

You can kill the process at any time with CTRL+C - the processor will attempt to shutdown cleanly and will wait up to :shutdown_timeout seconds for all actively running jobs to finish before killing them.

Note: When running in production, its recommended that use a process supervisor such as foreman, systemd, upstart, daemontools, launchd, runit, ect.

Performance

AWS SQS ActiveJob is a lightweight and performant queueing backend. Benchmark performed using: Ruby MRI 2.6.5,
shoryuken 5.0.5, aws-sdk-rails 3.3.1 and aws-sdk-sqs 1.34.0 on a 2015 Macbook Pro dual-core i7 with 16GB ram.

AWS SQS ActiveJob (default settings): Throughput 119.1 jobs/sec Shoryuken (default settings): Throughput 76.8 jobs/sec

Serverless workers: processing activejobs using AWS Lambda

Rather than managing the worker processes yourself, you can use Lambda with an SQS Trigger. With Lambda Container Image Support and the lambda handler provided with aws-sdk-rails its easy to use lambda to run ActiveJobs for your dockerized rails app (see below for some tips). All you need to do is:

  1. include the aws_lambda_ric gem
  2. Push your image to ecr
  3. Create a lambda function from your image (see the lambda docs for details).
  4. Add an SQS Trigger for the queue(s) you want to process jobs from.
  5. Set the ENTRYPOINT to /usr/local/bundle/bin/aws_lambda_ric and the CMD to config/environment.Aws::Rails::SqsActiveJob.lambda_job_handler - this will load Rails and then use the lambda handler provided by aws-sdk-rails. You can do this either as function config or in your Dockerfile.

There are a few limitations/requirements for lambda container images: the default lambda user must be able to read all the files and the image must be able to run on a read only file system. You may need to disable bootsnap, set a HOME env variable and set the logger to STDOUT (which lambda will record to cloudwatch for you).

You can use the RAILS_ENV to control environment. If you need to execute specific configuration in the lambda, you can create a ruby file and use it as your entrypoint:

# app.rb
# some custom config

require_relative 'config/environment' # load rails

# Rails.config.custom....
# Aws::Rails::SqsActiveJob.config....

# no need to write a handler yourself here, as long as
# aws-sdk-rails is loaded, you can still use the
# Aws::Rails::SqsActiveJob.lambda_job_handler

# To use this file, set CMD:  app.Aws::Rails::SqsActiveJob.lambda_job_handler

Elastic Beanstalk workers: processing activejobs using worker environments

Another option for processing jobs without managing the worker process is hosting the application in a scalable Elastic Beanstalk worker environment. This SDK includes Rack middleware that can be added conditionally and which will process requests from the SQS Daemon provided with each worker instance. The middleware will forward each request and parameters to their appropriate jobs.

To add the middleware on application startup, set the AWS_PROCESS_BEANSTALK_WORKER_REQUESTS environment variable to true in the worker environment configuration.

To protect against forgeries, daemon requests will only be processed if they originate from localhost or the Docker host.

Periodic (scheduled) jobs are also supported with this approach without requiring any additional dependencies. Elastic Beanstalk workers support the addition of a cron.yaml file in the application root to configure this.

Example:

version: 1
cron:
 - name: "MyApplicationJob"
   url: "/"
   schedule: "0 */12 * * *"

Where 'name' must be the case-sensitive class name of the job.

Configuration

For a complete list of configuration options see the Aws::Rails::SqsActiveJob::Configuration documentation.

You can configure AWS SQS Active Job either through the yml file or through code in your config/.rb or initializers.

For file based configuration, you can use either:

  1. config/aws_sqs_active_job/<RAILS_ENV>.yml
  2. config/aws_sqs_active_job.yml

The yml file supports ERB.

To configure in code:

Aws::Rails::SqsActiveJob.configure do |config|
  config.logger = ActiveSupport::Logger.new(STDOUT)
  config.max_messages = 5
  config.client = Aws::SQS::Client.new(region: 'us-east-1')
end

Using FIFO queues

If the order in which your jobs executes is important, consider using a FIFO Queue. A FIFO queue ensures that messages are processed in the order they were sent (First-In-First-Out) and exactly-once processing (ensuring duplicates are never introduced into the queue). To use a fifo queue, simply set the queue url (which will end in ".fifo") in your config.

When using FIFO queues, jobs will NOT be processed concurrently by the poller to ensure the correct ordering. Additionally, all jobs on a FIFO queue will be queued synchronously, even if you have configured the amazon_sqs_async adapter.

Message Deduplication ID

FIFO queues support Message deduplication ID, which is the token used for deduplication of sent messages. If a message with a particular message deduplication ID is sent successfully, any messages sent with the same message deduplication ID are accepted successfully but aren't delivered during the 5-minute deduplication interval.

Customize Deduplication keys

If necessary, the deduplication key used to create the message deduplication ID can be customized:

Aws::Rails::SqsActiveJob.configure do |config|
  config.excluded_deduplication_keys = [:job_class, :arguments]
end

# Or to set deduplication keys to exclude for a single job:
class YourJob < ApplicationJob
  include Aws::Rails::SqsActiveJob
  deduplicate_without :job_class, :arguments
  #...
end

By default, the following keys are used for deduplication keys:

job_class, provider_job_id, queue_name, priority, arguments, executions, exception_executions, locale, timezone, enqueued_at

Note that job_id is NOT included in deduplication keys because it is unique for each initialization of the job, and the run-once behavior must be guaranteed for ActiveJob retries. Even without setting job_id, it is implicitly excluded from deduplication keys.

Message Group IDs

FIFO queues require a message group id to be provided for the job. It is determined by:

  1. Calling message_group_id on the job if it is defined
  2. If message_group_id is not defined or the result is nil, the default value will be used. You can optionally specify a custom value in your config as the default that will be used by all jobs.

AWS Record Generators

This package also pulls in the `aws-record` gem and provides generators for creating models and a rake task for performing table config migrations.

Setup

You can either invoke the generator by calling rails g aws_record:model ...

If DynamoDB will be the only datastore you plan on using you can also set aws-record-generator to be your project's default orm with

config.generators do |g|
  g.orm :aws_record
end

Which will cause aws_record:model to be invoked by the Rails model generator.

Generating a model

Generating a model can be as simple as: rails g aws_record:model Forum --table-config primary:10-5 aws-record-generator will automatically create a uuid:hash_key field for you, and a table config with the provided r/w units

# app/models/forum.rb

require 'aws-record'

class Forum
  include Aws::Record

  string_attr :uuid, hash_key: true
end

# db/table_config/forum_config.rb

require 'aws-record'

module ModelTableConfig
  def self.config
    Aws::Record::TableConfig.define do |t|
      t.model_class Forum

      t.read_capacity_units 10
      t.write_capacity_units 5
    end
  end
end

More complex models can be created by adding more fields to the model as well as other options:

rails g aws_record Forum post_id:rkey author_username post_title post_body tags:sset:default_value{Set.new}

# app/models/forum.rb

require 'aws-record'

class Forum
  include Aws::Record

  string_attr :uuid, hash_key: true
  string_attr :post_id, range_key: true
  string_attr :author_username
  string_attr :post_title
  string_attr :post_body
  string_set_attr :tags, default_value: Set.new
end

# db/table_config/forum_config.rb
# ...

Finally you can attach a variety of options to your fields, and even ActiveModel validations to the models:

rails g aws_record:model Forum forum_uuid:hkey post_id:rkey author_username post_title post_body tags:sset:default_value{Set.new} created_at:datetime:db_attr_name{PostCreatedAtTime} moderation:boolean:default_value{false} --table-config=primary:5-2 AuthorIndex:12-14 --required=post_title --length-validations=post_body:50-1000 --gsi=AuthorIndex:hkey{author_username}

Which results in the following files being generated:

# app/models/forum.rb

require 'aws-record'
require 'active_model'

class Forum
  include Aws::Record
  include ActiveModel::Validations

  string_attr :forum_uuid, hash_key: true
  string_attr :post_id, range_key: true
  string_attr :author_username
  string_attr :post_title
  string_attr :post_body
  string_set_attr :tags, default_value: Set.new
  datetime_attr :created_at, database_attribute_name: "PostCreatedAtTime"
  boolean_attr :moderation, default_value: false

  global_secondary_index(
    :AuthorIndex,
    hash_key: :author_username,
    projection: {
      projection_type: "ALL"
    }
  )
  validates_presence_of :post_title
  validates_length_of :post_body, within: 50..1000
end

# db/table_config/forum_config.rb
# ...

To migrate your new models and begin using them you can run the provided rake task: rails aws_record:migrate

Docs

The syntax for creating an aws-record model follows:

rails generate aws_record:model NAME [field[:type][:opts]...] [options]

The possible field types are:

Field Name aws-record attribute type
bool | boolean :boolean_attr
date :date_attr
datetime :datetime_attr
float :float_attr
int | integer :integer_attr
list :list_attr
map :map_attr
num_set | numeric_set | nset :numeric_set_attr
string_set | s_set | sset :string_set_attr
string :string_attr

If a type is not provided, it will assume the field is of type :string_attr.

Additionally a number of options may be attached as a comma separated list to the field:

Field Option Name aws-record option
hkey marks an attribute as a hash_key
rkey marks an attribute as a range_key
persist_nil will persist nil values in a attribute
db_attr_name{NAME} sets a secondary name for an attribute, these must be unique across attribute names
ddb_type{S|N|B|BOOL|SS|NS|BS|M|L} sets the dynamo_db_type for an attribute
default_value{Object} sets the default value for an attribute

The standard rules apply for using options in a model. Additional reading can be found here

Command Option Names Purpose
[--skip-namespace], [--no-skip-namespace] Skip namespace (affects only isolated applications)
[--disable-mutation-tracking], [--no-disable-mutation-tracking] Disables dirty tracking
[--timestamps], [--no-timestamps] Adds created, updated timestamps to the model
--table-config=primary:R-W [SecondaryIndex1:R-W]... Declares the r/w units for the model as well as any secondary indexes
[--gsi=name:hkey{ field_name }[,rkey{ field_name },proj_type{ ALL|KEYS_ONLY|INCLUDE }]...] Allows for the declaration of secondary indexes
[--required=field1...] A list of attributes that are required for an instance of the model
[--length-validations=field1:MIN-MAX...] Validations on the length of attributes in a model
[--table-name=name] Sets the name of the table in DynamoDB, if different than the model name
[--skip-table-config] Doesn't generate a table config for the model
[--password-digest] Adds a password field (note that you must have bcrypt has a dependency) that automatically hashes and manages the model password

The included rake task aws_record:migrate will run all of the migrations in app/db/table_config

More Repositories

1

aws-cli

Universal Command Line Interface for Amazon Web Services
Python
14,304
star
2

chalice

Python Serverless Microframework for AWS
Python
10,654
star
3

aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
JavaScript
10,440
star
4

amazon-sagemaker-examples

Example πŸ““ Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
Jupyter Notebook
9,542
star
5

serverless-application-model

The AWS Serverless Application Model (AWS SAM) transform is a AWS CloudFormation macro that transforms SAM templates into CloudFormation templates.
Python
9,342
star
6

aws-sdk-js

AWS SDK for JavaScript in the browser and Node.js
JavaScript
7,476
star
7

aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
Python
6,506
star
8

aws-sdk-php

Official repository of the AWS SDK for PHP (@awsforphp)
PHP
5,886
star
9

containers-roadmap

This is the public roadmap for AWS container services (ECS, ECR, Fargate, and EKS).
Shell
5,164
star
10

karpenter

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
Go
4,615
star
11

s2n-tls

An implementation of the TLS/SSL protocols
C
4,465
star
12

aws-sdk-java

The official AWS SDK for Java 1.x. The AWS SDK for Java 2.x is available here: https://github.com/aws/aws-sdk-java-v2/
Java
4,117
star
13

aws-lambda-go

Libraries, samples and tools to help Go developers develop AWS Lambda functions.
Go
3,624
star
14

aws-sdk-pandas

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Python
3,537
star
15

copilot-cli

The AWS Copilot CLI is a tool for developers to build, release and operate production ready containerized applications on AWS App Runner or Amazon ECS on AWS Fargate.
Go
3,488
star
16

aws-sdk-ruby

The official AWS SDK for Ruby.
Ruby
3,462
star
17

amazon-freertos

DEPRECATED - See README.md
C
2,535
star
18

aws-sdk-go-v2

AWS SDK for the Go programming language.
Go
2,518
star
19

aws-sdk-js-v3

Modularized AWS SDK for JavaScript.
TypeScript
2,476
star
20

jsii

jsii allows code in any language to naturally interact with JavaScript classes. It is the technology that enables the AWS Cloud Development Kit to deliver polyglot libraries from a single codebase!
TypeScript
2,371
star
21

sagemaker-python-sdk

A library for training and deploying machine learning models on Amazon SageMaker
Python
2,095
star
22

amazon-vpc-cni-k8s

Networking plugin repository for pod networking in Kubernetes using Elastic Network Interfaces on AWS
Go
2,071
star
23

aws-eks-best-practices

A best practices guide for day 2 operations, including operational excellence, security, reliability, performance efficiency, and cost optimization.
Python
2,022
star
24

amazon-ecs-agent

Amazon Elastic Container Service Agent
Go
2,005
star
25

lumberyard

Amazon Lumberyard is a free AAA game engine deeply integrated with AWS and Twitch – with full source.
C++
1,965
star
26

aws-sdk-net

The official AWS SDK for .NET. For more information on the AWS SDK for .NET, see our web site:
1,945
star
27

eks-anywhere

Run Amazon EKS on your own infrastructure πŸš€
Go
1,899
star
28

aws-sdk-java-v2

The official AWS SDK for Java - Version 2
Java
1,822
star
29

aws-sdk-cpp

AWS SDK for C++
1,779
star
30

amazon-ecs-cli

The Amazon ECS CLI enables users to run their applications on ECS/Fargate using the Docker Compose file format, quickly provision resources, push/pull images in ECR, and monitor running applications on ECS/Fargate.
Go
1,725
star
31

aws-sdk-php-laravel

A Laravel 5+ (and 4) service provider for the AWS SDK for PHP
PHP
1,589
star
32

serverless-java-container

A Java wrapper to run Spring, Spring Boot, Jersey, and other apps inside AWS Lambda.
Java
1,483
star
33

aws-node-termination-handler

Gracefully handle EC2 instance shutdown within Kubernetes
Go
1,443
star
34

aws-lambda-dotnet

Libraries, samples and tools to help .NET Core developers develop AWS Lambda functions.
C#
1,430
star
35

aws-fpga

Official repository of the AWS EC2 FPGA Hardware and Software Development Kit
VHDL
1,380
star
36

eks-distro

Amazon EKS Distro (EKS-D) is a Kubernetes distribution based on and used by Amazon Elastic Kubernetes Service (EKS) to create reliable and secure Kubernetes clusters.
Shell
1,263
star
37

eks-charts

Amazon EKS Helm chart repository
Mustache
1,184
star
38

s2n-quic

An implementation of the IETF QUIC protocol
Rust
1,152
star
39

aws-toolkit-vscode

CodeWhisperer, CodeCatalyst, Local Lambda debug, SAM/CFN syntax, ECS Terminal, AWS resources
TypeScript
1,150
star
40

opsworks-cookbooks

Chef Cookbooks for the AWS OpsWorks Service
Ruby
1,058
star
41

aws-codebuild-docker-images

Official AWS CodeBuild repository for managed Docker images http://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref.html
Dockerfile
1,032
star
42

amazon-ssm-agent

An agent to enable remote management of your EC2 instances, on-premises servers, or virtual machines (VMs).
Go
975
star
43

aws-iot-device-sdk-js

SDK for connecting to AWS IoT from a device using JavaScript/Node.js
JavaScript
957
star
44

aws-iot-device-sdk-embedded-C

SDK for connecting to AWS IoT from a device using embedded C.
C
926
star
45

aws-health-tools

The samples provided in AWS Health Tools can help users to build automation and customized alerting in response to AWS Health events.
Python
887
star
46

aws-graviton-getting-started

Helping developers to use AWS Graviton2, Graviton3, and Graviton4 processors which power the 6th, 7th, and 8th generation of Amazon EC2 instances (C6g[d], M6g[d], R6g[d], T4g, X2gd, C6gn, I4g, Im4gn, Is4gen, G5g, C7g[d][n], M7g[d], R7g[d], R8g).
Python
850
star
47

aws-app-mesh-examples

AWS App Mesh is a service mesh that you can use with your microservices to manage service to service communication.
Shell
844
star
48

deep-learning-containers

AWS Deep Learning Containers (DLCs) are a set of Docker images for training and serving models in TensorFlow, TensorFlow 2, PyTorch, and MXNet.
Python
800
star
49

aws-parallelcluster

AWS ParallelCluster is an AWS supported Open Source cluster management tool to deploy and manage HPC clusters in the AWS cloud.
Python
782
star
50

aws-lambda-runtime-interface-emulator

Go
771
star
51

aws-toolkit-jetbrains

AWS Toolkit for JetBrains - a plugin for interacting with AWS from JetBrains IDEs
Kotlin
735
star
52

graph-notebook

Library extending Jupyter notebooks to integrate with Apache TinkerPop, openCypher, and RDF SPARQL.
Jupyter Notebook
706
star
53

aws-iot-device-sdk-python

SDK for connecting to AWS IoT from a device using Python.
Python
670
star
54

amazon-chime-sdk-js

A JavaScript client library for integrating multi-party communications powered by the Amazon Chime service.
TypeScript
655
star
55

amazon-ec2-instance-selector

A CLI tool and go library which recommends instance types based on resource criteria like vcpus and memory
Go
642
star
56

studio-lab-examples

Example notebooks for working with SageMaker Studio Lab. Sign up for an account at the link below!
Jupyter Notebook
625
star
57

aws-secretsmanager-agent

The AWS Secrets Manager Agent is a local HTTP service that you can install and use in your compute environments to read secrets from Secrets Manager and cache them in memory.
Rust
584
star
58

event-ruler

Event Ruler is a Java library that allows matching many thousands of Events per second to any number of expressive and sophisticated rules.
Java
564
star
59

aws-mwaa-local-runner

This repository provides a command line interface (CLI) utility that replicates an Amazon Managed Workflows for Apache Airflow (MWAA) environment locally.
Shell
553
star
60

amazon-eks-pod-identity-webhook

Amazon EKS Pod Identity Webhook
Go
534
star
61

aws-lambda-java-libs

Official mirror for interface definitions and helper classes for Java code running on the AWS Lambda platform.
C++
518
star
62

aws-lambda-base-images

506
star
63

aws-appsync-community

The AWS AppSync community
HTML
495
star
64

sagemaker-training-toolkit

Train machine learning models within a 🐳 Docker container using 🧠 Amazon SageMaker.
Python
493
star
65

dotnet

GitHub home for .NET development on AWS
487
star
66

aws-cdk-rfcs

RFCs for the AWS CDK
JavaScript
476
star
67

aws-sam-cli-app-templates

Python
472
star
68

aws-elastic-beanstalk-cli-setup

Simplified EB CLI installation mechanism.
Python
453
star
69

amazon-cloudwatch-agent

CloudWatch Agent enables you to collect and export host-level metrics and logs on instances running Linux or Windows server.
Go
403
star
70

secrets-store-csi-driver-provider-aws

The AWS provider for the Secrets Store CSI Driver allows you to fetch secrets from AWS Secrets Manager and AWS Systems Manager Parameter Store, and mount them into Kubernetes pods.
Go
393
star
71

amazon-braket-examples

Example notebooks that show how to apply quantum computing in Amazon Braket.
Python
376
star
72

aws-for-fluent-bit

The source of the amazon/aws-for-fluent-bit container image
Shell
375
star
73

aws-pdk

The AWS PDK provides building blocks for common patterns together with development tools to manage and build your projects.
TypeScript
361
star
74

aws-extensions-for-dotnet-cli

Extensions to the dotnet CLI to simplify the process of building and publishing .NET Core applications to AWS services
C#
346
star
75

aws-sdk-php-symfony

PHP
346
star
76

aws-app-mesh-roadmap

AWS App Mesh is a service mesh that you can use with your microservices to manage service to service communication
344
star
77

aws-lambda-builders

Python library to compile, build & package AWS Lambda functions for several runtimes & framework
Python
337
star
78

aws-iot-device-sdk-python-v2

Next generation AWS IoT Client SDK for Python using the AWS Common Runtime
Python
335
star
79

constructs

Define composable configuration models through code
TypeScript
332
star
80

pg_tle

Framework for building trusted language extensions for PostgreSQL
C
329
star
81

graph-explorer

React-based web application that enables users to visualize both property graph and RDF data and explore connections between data without having to write graph queries.
TypeScript
321
star
82

aws-codedeploy-agent

Host Agent for AWS CodeDeploy
Ruby
316
star
83

aws-sdk-ruby-record

Official repository for the aws-record gem, an abstraction for Amazon DynamoDB.
Ruby
313
star
84

aws-ops-wheel

The AWS Ops Wheel is a randomizer that biases for options that haven’t come up recently; you can also outright cheat and specify the next result to be generated.
JavaScript
308
star
85

aws-xray-sdk-python

AWS X-Ray SDK for the Python programming language
Python
304
star
86

sagemaker-inference-toolkit

Serve machine learning models within a 🐳 Docker container using 🧠 Amazon SageMaker.
Python
303
star
87

efs-utils

Utilities for Amazon Elastic File System (EFS)
Python
286
star
88

amazon-ivs-react-native-player

A React Native wrapper for the Amazon IVS iOS and Android player SDKs.
TypeScript
286
star
89

sagemaker-spark

A Spark library for Amazon SageMaker.
Scala
282
star
90

apprunner-roadmap

This is the public roadmap for AWS App Runner.
280
star
91

aws-xray-sdk-go

AWS X-Ray SDK for the Go programming language.
Go
274
star
92

aws-toolkit-eclipse

(End of life: May 31, 2023) AWS Toolkit for Eclipse
Java
273
star
93

elastic-beanstalk-roadmap

AWS Elastic Beanstalk roadmap
272
star
94

aws-logging-dotnet

.NET Libraries for integrating Amazon CloudWatch Logs with popular .NET logging libraries
C#
271
star
95

sagemaker-tensorflow-training-toolkit

Toolkit for running TensorFlow training scripts on SageMaker. Dockerfiles used for building SageMaker TensorFlow Containers are at https://github.com/aws/deep-learning-containers.
Python
270
star
96

aws-lc-rs

aws-lc-rs is a cryptographic library using AWS-LC for its cryptographic operations. The library strives to be API-compatible with the popular Rust library named ring.
Rust
263
star
97

elastic-load-balancing-tools

AWS Elastic Load Balancing Tools
Java
262
star
98

aws-step-functions-data-science-sdk-python

Step Functions Data Science SDK for building machine learning (ML) workflows and pipelines on AWS
Python
261
star
99

amazon-braket-sdk-python

A Python SDK for interacting with quantum devices on Amazon Braket
Python
254
star
100

aws-xray-sdk-node

The official AWS X-Ray SDK for Node.js.
JavaScript
248
star