• Stars
    star
    850
  • Rank 53,628 (Top 2 %)
  • Language
    Python
  • License
    Other
  • Created almost 5 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Helping developers to use AWS Graviton2, Graviton3, and Graviton4 processors which power the 6th, 7th, and 8th generation of Amazon EC2 instances (C6g[d], M6g[d], R6g[d], T4g, X2gd, C6gn, I4g, Im4gn, Is4gen, G5g, C7g[d][n], M7g[d], R7g[d], R8g).

AWS Graviton Technical Guide

This repository provides technical guidance for users and developers using Amazon EC2 instances powered by AWS Graviton processors (including the latest generation Graviton3 processors). While it calls out specific features of the Graviton processors themselves, this repository is also generally useful for anyone running code on Arm-based systems.

Contents

Transitioning to Graviton

If you are new to Graviton and want to understand how to identify target workloads, how to plan a transition project, how to test your workloads on AWS Graviton and finally how deploy in production, please read the key considerations to take into account when transitioning workloads to AWS Graviton based Amazon EC2 instances.

Building for Graviton2, Graviton3 and Graviton3E

Processor Graviton2 Graviton3(E)
Instances M6g/M6gd, C6g/C6gd/C6gn, R6g/R6gd, T4g, X2gd, G5g, and Im4gn/Is4gen C7g/C7gn, M7g, R7g, and HPC7g
Core Neoverse-N1 Neoverse-V1
Frequency 2500MHz 2600MHz
Turbo supported No No
Instruction latencies Instruction Latencies Instruction Latencies
Interconnect CMN-600 CMN-650
Architecture revision ARMv8.2-a ARMv8.4-a
Additional features fp16, rcpc, dotprod, crypto sve, rng, bf16, int8, crypto
Recommended -mcpu flag neoverse-n1 neoverse-512tvb
RNG Instructions No Yes
SIMD instructions 2x Neon 128bit vectors 4x Neon 128bit vectors / 2x SVE 256bit
LSE (atomic mem operations) yes yes
Pointer Authentication no yes
Cores 64 64
L1 cache (per core) 64KB inst / 64KB data 64KB inst / 64KB data
L2 cache (per core) 1MB 1MB
LLC (shared) 32MB 32MB
DRAM 8x DDR4 8x DDR5
DDR Encryption yes yes

Optimizing for Graviton

Please refer to optimizing for general debugging and profiling information. For detailed checklists on optimizing and debugging performance on Graviton, see our performance runbook.

Different architectures and systems have differing capabilities, which means some tools you might be familiar with on one architecture don't have equivalent on AWS Graviton. Documented Monitoring Tools with some of these utilities.

Recent software updates relevant to Graviton

There is a huge amount of activity in the Arm software ecosystem and improvements are being made on a daily basis. As a general rule later versions of compilers, language runtimes, and applications should be used whenever possible. The table below includes known recent changes to popular packages that improve performance (if you know of others please let us know).

Package Version Improvements
bazel 3.4.1+ Pre-built bazel binary for Graviton/Arm64. See below for installation.
Cassandra 4.0+ Supports running on Java/Corretto 11, improving overall performance
FFmpeg 5.1+ Improved performance of libswscale by 50% with better NEON vectorization which improves the performance and scalability of FFmpeg multi-thread encoders. The changes are available in FFmpeg version 4.3, with further improvements to scaling and motion estimation available in 5.1. Additional improvements to both will be available in 5.2, which has not yet been released as of November 2022. For more information about FFmpeg on Graviton, read the blog post on AWS Open Source Blog, Optimized Video Encoding with FFmpeg on AWS Graviton Processors.
HAProxy 2.4+ A serious bug was fixed. Additionally, building with CPU=armv81 improves HAProxy performance by 4x so please rebuild your code with this flag.
MariaDB 10.4.14+ Default build now uses -moutline-atomics, general correctness bugs for Graviton fixed.
mongodb 4.2.15+ / 4.4.7+ / 5.0.0+ Improved performance on graviton, especially for internal JS engine. LSE support added in SERVER-56347.
MySQL 8.0.23+ Improved spinlock behavior, compiled with -moutline-atomics if compiler supports it.
PostgreSQL 15+ General scalability improvements plus additional improvements to spin-locks specifically for Arm64
.NET 5+ .NET 5 significantly improved performance for ARM64. Here's an associated AWS Blog with some performance results.
OpenH264 2.1.1+ Pre-built Cisco OpenH264 binary for Graviton/Arm64.
PCRE2 10.34+ Added NEON vectorization to PCRE's JIT to match first and pairs of characters. This may improve performance of matching by up to 8x. This fixed version of the library now is shipping with Ubuntu 20.04 and PHP 8.
PHP 7.4+ PHP 7.4 includes a number of performance improvements that increase perf by up to 30%
pip 19.3+ Enable installation of python wheel binaries on Graviton
PyTorch 2.0+ Optimize Inference latency and throughput on Graviton. AWS DLCs and python wheels are available.
ruby 3.0+ Enable arm64 optimizations that improve performance by as much as 40%. These changes have also been back-ported to the Ruby shipping with AmazonLinux2, Fedora, and Ubuntu 20.04.
Spark 3.0+ Supports running on Java/Corretto 11, improving overall performance.
zlib 1.2.8+ For the best performance on Graviton please use zlib-cloudflare.

Containers on Graviton

You can run Docker, Kubernetes, Amazon ECS, and Amazon EKS on Graviton. Amazon ECR supports multi-arch containers. Please refer to containers for information about running container-based workloads on Graviton.

Lambda on Graviton

AWS Lambda now allows you to configure new and existing functions to run on Arm-based AWS Graviton2 processors in addition to x86-based functions. Using this processor architecture option allows you to get up to 34% better price performance. Duration charges are 20 percent lower than the current pricing for x86 with millisecond granularity. This also applies to duration charges when using Provisioned Concurrency. Compute Savings Plans supports Lambda functions powered by Graviton2.

The Lambda page highlights some of the migration considerations and also provides some simple to deploy demos you can use to explore how to build and migrate to Lambda functions using Arm/Graviton2.

Operating Systems

Please check os.md for more information about which operating system to run on Graviton based instances.

Known issues and workarounds

Postgres

Postgres performance can be heavily impacted by not using LSE. Today, postgres binaries from distributions (e.g. Ubuntu) are not built with -moutline-atomics or -march=armv8.2-a which would enable LSE. Note: Amazon RDS for PostgreSQL isn't impacted by this.

In November 2021 PostgreSQL started to distribute Ubuntu 20.04 packages optimized with -moutline-atomics. For Ubuntu 20.04, we recommend using the PostgreSQL PPA instead of the packages distributed by Ubuntu Focal. Please follow the instructions to set up the PostgreSQL PPA.

Python installation on some Linux distros

The default installation of pip on some Linux distributions is old (<19.3) to install binary wheel packages released for Graviton. To work around this, it is recommended to upgrade your pip installation using:

sudo python3 -m pip install --upgrade pip

Bazel on Linux

The Bazel build tool now releases a pre-built binary for arm64. As of October 2020, this is not available in their custom Debian repo, and Bazel does not officially provide an RPM. Instead, we recommend using the Bazelisk installer, which will replace your bazel command and keep bazel up to date.

Below is an example using the latest Arm binary release of Bazelisk as of October 2020:

wget https://github.com/bazelbuild/bazelisk/releases/download/v1.7.1/bazelisk-linux-arm64
chmod +x bazelisk-linux-arm64
sudo mv bazelisk-linux-arm64 /usr/local/bin/bazel
bazel

Bazelisk itself should not require further updates, as its only purpose is to keep Bazel updated.

zlib on Linux

Linux distributions, in general, use the original zlib without any optimizations. zlib-cloudflare has been updated to provide better and faster compression on Arm and x86. To use zlib-cloudflare:

git clone https://github.com/cloudflare/zlib.git
cd zlib
./configure --prefix=$HOME
make
make install

Make sure to have the full path to your lib at $HOME/lib in /etc/ld.so.conf and run ldconfig.

For users of OpenJDK, which is dynamically linked to the system zlib, you can set LD_LIBRARY_PATH to point to the directory where your newly built version of zlib-cloudflare is located or load that library with LD_PRELOAD.

You can check the libz that JDK is dynamically linked against with:

$ ldd /Java/jdk-11.0.8/lib/libzip.so | grep libz
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007ffff7783000)

Currently, users of Amazon Corretto cannot link against zlib-cloudflare.

Additional resources

Feedback? [email protected]

More Repositories

1

aws-cli

Universal Command Line Interface for Amazon Web Services
Python
14,304
star
2

chalice

Python Serverless Microframework for AWS
Python
10,654
star
3

aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
JavaScript
10,440
star
4

amazon-sagemaker-examples

Example ๐Ÿ““ Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using ๐Ÿง  Amazon SageMaker.
Jupyter Notebook
9,542
star
5

serverless-application-model

The AWS Serverless Application Model (AWS SAM) transform is a AWS CloudFormation macro that transforms SAM templates into CloudFormation templates.
Python
9,342
star
6

aws-sdk-js

AWS SDK for JavaScript in the browser and Node.js
JavaScript
7,476
star
7

aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
Python
6,506
star
8

aws-sdk-php

Official repository of the AWS SDK for PHP (@awsforphp)
PHP
5,886
star
9

containers-roadmap

This is the public roadmap for AWS container services (ECS, ECR, Fargate, and EKS).
Shell
5,164
star
10

karpenter

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
Go
4,615
star
11

s2n-tls

An implementation of the TLS/SSL protocols
C
4,465
star
12

aws-sdk-java

The official AWS SDK for Java 1.x. The AWS SDK for Java 2.x is available here: https://github.com/aws/aws-sdk-java-v2/
Java
4,117
star
13

aws-lambda-go

Libraries, samples and tools to help Go developers develop AWS Lambda functions.
Go
3,624
star
14

aws-sdk-pandas

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Python
3,537
star
15

copilot-cli

The AWS Copilot CLI is a tool for developers to build, release and operate production ready containerized applications on AWS App Runner or Amazon ECS on AWS Fargate.
Go
3,488
star
16

aws-sdk-ruby

The official AWS SDK for Ruby.
Ruby
3,462
star
17

amazon-freertos

DEPRECATED - See README.md
C
2,535
star
18

aws-sdk-go-v2

AWS SDK for the Go programming language.
Go
2,518
star
19

aws-sdk-js-v3

Modularized AWS SDK for JavaScript.
TypeScript
2,476
star
20

jsii

jsii allows code in any language to naturally interact with JavaScript classes. It is the technology that enables the AWS Cloud Development Kit to deliver polyglot libraries from a single codebase!
TypeScript
2,371
star
21

sagemaker-python-sdk

A library for training and deploying machine learning models on Amazon SageMaker
Python
2,095
star
22

amazon-vpc-cni-k8s

Networking plugin repository for pod networking in Kubernetes using Elastic Network Interfaces on AWS
Go
2,071
star
23

aws-eks-best-practices

A best practices guide for day 2 operations, including operational excellence, security, reliability, performance efficiency, and cost optimization.
Python
2,022
star
24

amazon-ecs-agent

Amazon Elastic Container Service Agent
Go
2,005
star
25

lumberyard

Amazon Lumberyard is a free AAA game engine deeply integrated with AWS and Twitch โ€“ with full source.
C++
1,965
star
26

aws-sdk-net

The official AWS SDK for .NET. For more information on the AWS SDK for .NET, see our web site:
1,945
star
27

eks-anywhere

Run Amazon EKS on your own infrastructure ๐Ÿš€
Go
1,899
star
28

aws-sdk-java-v2

The official AWS SDK for Java - Version 2
Java
1,822
star
29

aws-sdk-cpp

AWS SDK for C++
1,779
star
30

amazon-ecs-cli

The Amazon ECS CLI enables users to run their applications on ECS/Fargate using the Docker Compose file format, quickly provision resources, push/pull images in ECR, and monitor running applications on ECS/Fargate.
Go
1,725
star
31

aws-sdk-php-laravel

A Laravel 5+ (and 4) service provider for the AWS SDK for PHP
PHP
1,589
star
32

serverless-java-container

A Java wrapper to run Spring, Spring Boot, Jersey, and other apps inside AWS Lambda.
Java
1,483
star
33

aws-node-termination-handler

Gracefully handle EC2 instance shutdown within Kubernetes
Go
1,443
star
34

aws-lambda-dotnet

Libraries, samples and tools to help .NET Core developers develop AWS Lambda functions.
C#
1,430
star
35

aws-fpga

Official repository of the AWS EC2 FPGA Hardware and Software Development Kit
VHDL
1,380
star
36

eks-distro

Amazon EKS Distro (EKS-D) is a Kubernetes distribution based on and used by Amazon Elastic Kubernetes Service (EKS) to create reliable and secure Kubernetes clusters.
Shell
1,263
star
37

eks-charts

Amazon EKS Helm chart repository
Mustache
1,184
star
38

s2n-quic

An implementation of the IETF QUIC protocol
Rust
1,152
star
39

aws-toolkit-vscode

CodeWhisperer, CodeCatalyst, Local Lambda debug, SAM/CFN syntax, ECS Terminal, AWS resources
TypeScript
1,150
star
40

opsworks-cookbooks

Chef Cookbooks for the AWS OpsWorks Service
Ruby
1,058
star
41

aws-codebuild-docker-images

Official AWS CodeBuild repository for managed Docker images http://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref.html
Dockerfile
1,032
star
42

amazon-ssm-agent

An agent to enable remote management of your EC2 instances, on-premises servers, or virtual machines (VMs).
Go
975
star
43

aws-iot-device-sdk-js

SDK for connecting to AWS IoT from a device using JavaScript/Node.js
JavaScript
957
star
44

aws-iot-device-sdk-embedded-C

SDK for connecting to AWS IoT from a device using embedded C.
C
926
star
45

aws-health-tools

The samples provided in AWS Health Tools can help users to build automation and customized alerting in response to AWS Health events.
Python
887
star
46

aws-app-mesh-examples

AWS App Mesh is a service mesh that you can use with your microservices to manage service to service communication.
Shell
844
star
47

deep-learning-containers

AWS Deep Learning Containers (DLCs) are a set of Docker images for training and serving models in TensorFlow, TensorFlow 2, PyTorch, and MXNet.
Python
800
star
48

aws-parallelcluster

AWS ParallelCluster is an AWS supported Open Source cluster management tool to deploy and manage HPC clusters in the AWS cloud.
Python
782
star
49

aws-lambda-runtime-interface-emulator

Go
771
star
50

aws-toolkit-jetbrains

AWS Toolkit for JetBrains - a plugin for interacting with AWS from JetBrains IDEs
Kotlin
735
star
51

graph-notebook

Library extending Jupyter notebooks to integrate with Apache TinkerPop, openCypher, and RDF SPARQL.
Jupyter Notebook
706
star
52

aws-iot-device-sdk-python

SDK for connecting to AWS IoT from a device using Python.
Python
670
star
53

amazon-chime-sdk-js

A JavaScript client library for integrating multi-party communications powered by the Amazon Chime service.
TypeScript
655
star
54

amazon-ec2-instance-selector

A CLI tool and go library which recommends instance types based on resource criteria like vcpus and memory
Go
642
star
55

studio-lab-examples

Example notebooks for working with SageMaker Studio Lab. Sign up for an account at the link below!
Jupyter Notebook
625
star
56

aws-secretsmanager-agent

The AWS Secrets Manager Agent is a local HTTP service that you can install and use in your compute environments to read secrets from Secrets Manager and cache them in memory.
Rust
584
star
57

event-ruler

Event Ruler is a Java library that allows matching many thousands of Events per second to any number of expressive and sophisticated rules.
Java
564
star
58

aws-sdk-rails

Official repository for the aws-sdk-rails gem, which integrates the AWS SDK for Ruby with Ruby on Rails.
Ruby
554
star
59

aws-mwaa-local-runner

This repository provides a command line interface (CLI) utility that replicates an Amazon Managed Workflows for Apache Airflow (MWAA) environment locally.
Shell
553
star
60

amazon-eks-pod-identity-webhook

Amazon EKS Pod Identity Webhook
Go
534
star
61

aws-lambda-java-libs

Official mirror for interface definitions and helper classes for Java code running on the AWS Lambda platform.
C++
518
star
62

aws-lambda-base-images

506
star
63

aws-appsync-community

The AWS AppSync community
HTML
495
star
64

sagemaker-training-toolkit

Train machine learning models within a ๐Ÿณ Docker container using ๐Ÿง  Amazon SageMaker.
Python
493
star
65

dotnet

GitHub home for .NET development on AWS
487
star
66

aws-cdk-rfcs

RFCs for the AWS CDK
JavaScript
476
star
67

aws-sam-cli-app-templates

Python
472
star
68

aws-elastic-beanstalk-cli-setup

Simplified EB CLI installation mechanism.
Python
453
star
69

amazon-cloudwatch-agent

CloudWatch Agent enables you to collect and export host-level metrics and logs on instances running Linux or Windows server.
Go
403
star
70

secrets-store-csi-driver-provider-aws

The AWS provider for the Secrets Store CSI Driver allows you to fetch secrets from AWS Secrets Manager and AWS Systems Manager Parameter Store, and mount them into Kubernetes pods.
Go
393
star
71

amazon-braket-examples

Example notebooks that show how to apply quantum computing in Amazon Braket.
Python
376
star
72

aws-for-fluent-bit

The source of the amazon/aws-for-fluent-bit container image
Shell
375
star
73

aws-pdk

The AWS PDK provides building blocks for common patterns together with development tools to manage and build your projects.
TypeScript
361
star
74

aws-extensions-for-dotnet-cli

Extensions to the dotnet CLI to simplify the process of building and publishing .NET Core applications to AWS services
C#
346
star
75

aws-sdk-php-symfony

PHP
346
star
76

aws-app-mesh-roadmap

AWS App Mesh is a service mesh that you can use with your microservices to manage service to service communication
344
star
77

aws-lambda-builders

Python library to compile, build & package AWS Lambda functions for several runtimes & framework
Python
337
star
78

aws-iot-device-sdk-python-v2

Next generation AWS IoT Client SDK for Python using the AWS Common Runtime
Python
335
star
79

constructs

Define composable configuration models through code
TypeScript
332
star
80

pg_tle

Framework for building trusted language extensions for PostgreSQL
C
329
star
81

graph-explorer

React-based web application that enables users to visualize both property graph and RDF data and explore connections between data without having to write graph queries.
TypeScript
321
star
82

aws-codedeploy-agent

Host Agent for AWS CodeDeploy
Ruby
316
star
83

aws-sdk-ruby-record

Official repository for the aws-record gem, an abstraction for Amazon DynamoDB.
Ruby
313
star
84

aws-ops-wheel

The AWS Ops Wheel is a randomizer that biases for options that havenโ€™t come up recently; you can also outright cheat and specify the next result to be generated.
JavaScript
308
star
85

aws-xray-sdk-python

AWS X-Ray SDK for the Python programming language
Python
304
star
86

sagemaker-inference-toolkit

Serve machine learning models within a ๐Ÿณ Docker container using ๐Ÿง  Amazon SageMaker.
Python
303
star
87

efs-utils

Utilities for Amazon Elastic File System (EFS)
Python
286
star
88

amazon-ivs-react-native-player

A React Native wrapper for the Amazon IVS iOS and Android player SDKs.
TypeScript
286
star
89

sagemaker-spark

A Spark library for Amazon SageMaker.
Scala
282
star
90

apprunner-roadmap

This is the public roadmap for AWS App Runner.
280
star
91

aws-xray-sdk-go

AWS X-Ray SDK for the Go programming language.
Go
274
star
92

aws-toolkit-eclipse

(End of life: May 31, 2023) AWS Toolkit for Eclipse
Java
273
star
93

elastic-beanstalk-roadmap

AWS Elastic Beanstalk roadmap
272
star
94

aws-logging-dotnet

.NET Libraries for integrating Amazon CloudWatch Logs with popular .NET logging libraries
C#
271
star
95

sagemaker-tensorflow-training-toolkit

Toolkit for running TensorFlow training scripts on SageMaker. Dockerfiles used for building SageMaker TensorFlow Containers are at https://github.com/aws/deep-learning-containers.
Python
270
star
96

aws-lc-rs

aws-lc-rs is a cryptographic library using AWS-LC for its cryptographic operations. The library strives to be API-compatible with the popular Rust library named ring.
Rust
263
star
97

elastic-load-balancing-tools

AWS Elastic Load Balancing Tools
Java
262
star
98

aws-step-functions-data-science-sdk-python

Step Functions Data Science SDK for building machine learning (ML) workflows and pipelines on AWS
Python
261
star
99

amazon-braket-sdk-python

A Python SDK for interacting with quantum devices on Amazon Braket
Python
254
star
100

aws-xray-sdk-node

The official AWS X-Ray SDK for Node.js.
JavaScript
248
star