• Stars
    star
    102
  • Rank 335,584 (Top 7 %)
  • Language
    Python
  • License
    MIT License
  • Created over 7 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

An easy interface to query the EC2 metadata API, with caching.

ec2-metadata

https://img.shields.io/github/actions/workflow/status/adamchainz/ec2-metadata/main.yml?branch=main&style=for-the-badge https://img.shields.io/badge/Coverage-100%25-success?style=for-the-badge https://img.shields.io/pypi/v/ec2-metadata.svg?style=for-the-badge https://img.shields.io/badge/code%20style-black-000000.svg?style=for-the-badge pre-commit

An easy interface to query the EC2 metadata API (version 2), with caching.

A quick example:

>>> from ec2_metadata import ec2_metadata
>>> print(ec2_metadata.region)
us-east-1
>>> print(ec2_metadata.instance_id)
i-123456

Installation

Use pip:

python -m pip install ec2-metadata

Python 3.8 to 3.12 supported.


Working on a Django project? Improve your skills with one of my books.


Why?

boto came with a utility function to retrieve the instance metadata as a lazy loading dictionary, boto.utils.get_instance_metadata, but this has not been ported to boto3, as per this issue. I thought that rather than building a new version inside boto3 it would work well as a standalone library.

Instance Metadata Service Version 2

In November 2019, AWS released version 2 of the instance metadata service. It's more secure against Server Side Request Forgery (SSRF) attacks.

ec2-metadata now uses it exclusively. You can therefore consider disabling version 1, as per AWS' guide.

Note: Instance Metadata Service v2 has a default IP hop limit of 1. This can mean that you can see requests.exceptions.ReadTimeout errors from within Docker containers. To solve this, reconfigure your EC2 instanceโ€™s metadata options to allow three hops with aws ec2 modify-instance-metadata-options:

aws ec2 modify-instance-metadata-options  --instance-id <instance-id> --http-put-response-hop-limit 3

API

EC2Metadata(session=None)

A container that represents the data available on the EC2 metadata service. Attributes don't entirely correspond to the paths in the metadata service - they have been 'cleaned up'. You may also want to refer to the metadata service docs to understand the exact contents.

There's a singleton instance of it at the name ec2_metadata which should cover 90% of use cases. Use it like:

from ec2_metadata import ec2_metadata

ec2_metadata.region

The session argument, if provided, should be an instance of requests.Session, allowing you to customize the way requests are made.

Most of the attributes are cached, except where noted below. This is because they are mostly immutable, or at least require an instance stop to change. However some cached attributes do represent things that can change without an instance stop, but rarely do, such as network devices.

The caching is done with @cached_property, so they cache on first access. If you want to clear the cache of one attribute you can just del it:

del ec2_metadata.network_interfaces

To clear all, use the clear_all() method as per below.

account_id: str

The current AWS account ID, e.g. '123456789012'.

ami_id: str

The ID of the AMI used to launch the instance, e.g. 'ami-123456'.

autoscaling_target_lifecycle_state: str | None

Uncached. The target Auto Scaling lifecycle state that the instance is transitionioning to, or None if the instance is not in an autoscaling group. See AWS docs page Retrieve the target lifecycle state through instance metadata.

availability_zone: str

The name of the current AZ e.g. 'eu-west-1a'.

availability_zone_id: str | None

The unique, cross-account ID of the current AZ e.g. 'use1-az6'. See AWS docs page AZ IDs for your AWS resources.

ami_launch_index: int

The index of the instance in the launch request, zero-based, e.g. 0.

ami_manifest_path: str

The path to the AMI manifest file in Amazon S3, or '(unknown)' on EBS-backed AMI's.

clear_all() -> None

Clear all the cached attributes on the class, meaning their next access will re-fetch the data from the metadata API. This includes clearing the token used to authenticate with the service.

domain: str

The domain for AWS resources for the region. E.g. 'amazonaws.com' for the standard AWS regions and GovCloud (US), or 'amazonaws.com.cn' for China.

iam_info: IamInfoDict | None

A dictionary of data for the IAM role attached to the instance, or None if no role is attached. The dict has this type, based on what the metadata service returns:

class IamInfoDict(TypedDict):
    InstanceProfileArn: str
    InstanceProfileId: str
    LastUpdated: str

iam_security_credentials: IamSecurityCredentialsDict | None

A dictionary of data for the security credentials associated with the IAM role attached to the instance, or None if no role is attached. See the AWS docs section โ€œRetrieve security credentials from instance metadataโ€ for details. The dict has this type, based on that document:

class IamSecurityCredentialsDict(TypedDict):
    LastUpdated: str
    Type: str
    AccessKeyId: str
    SecretAccessKey: str
    Token: str
    Expiration: str

instance_action: str

Uncached. A state that notifies if the instance will reboot in preparation for bundling. See the AWS docs section โ€œInstance Metadata Categoriesโ€ for the valid values.

instance_id: str

The current instance's ID, e.g. 'i-123456'

instance_identity_document: InstanceIdentityDocumentDict

A dictionary of dynamic data about the instance. See the AWS docs page โ€œInstance Identity Documentsโ€ for an explanation of the contents. The dict has this type, based on that document:

class InstanceIdentityDocumentDict(TypedDict):
    accountId: str
    architecture: Literal["i386", "x86_64", "arm64"]
    availabilityZone: str
    billingProducts: list[str] | None
    marketplaceProductCodes: list[str] | None
    imageId: str
    instanceId: str
    instanceType: str
    kernelId: str | None
    pendingTime: str
    privateIp: str
    ramdiskId: str | None
    region: str
    version: str

instance_profile_arn: str | None

The ARN of the IAM role/instance profile attached to the instance, taken from iam_info, or None if no role is attached.

instance_profile_id: str | None

The ID of the IAM role/instance profile attached to the instance, taken from iam_info, or None if no role is attached.

instance_profile_name: str | None

The instance profile name, extracted from instance_profile_arn, or None if no role is attached.

instance_type: str

The current instance's type, e.g. 't2.nano'

kernel_id: str | None

The current instance's kernel ID, or None if it doesn't have one, e.g. 'aki-dc9ed9af'.

mac : str

The instance's MAC address, e.g. '0a:d2:ae:4d:f3:12'

network_interfaces: dict[str, NetworkInterface]

A dictionary of mac address to NetworkInterface, which represents the data available on a network interface - see below. E.g. {'01:23:45:67:89:ab': NetworkInterface('01:23:45:67:89:ab')}

partition: str

The AWS partition where the instance is running. E.g. 'aws' for the standard AWS regions, 'aws-us-gov' for GovCloud (US), or 'aws-cn' for China.

private_hostname : str

The private IPv4 DNS hostname of the instance, e.g. 'ip-172-30-0-0.eu-west-1.compute.internal' .

private_ipv4: str

The private IPv4 of the instance, e.g. '172.30.0.0'.

public_hostname : str | None

The public DNS hostname of the instance, or None if the instance is not public, e.g. 'ec2-1-2-3-4.compute-1.amazonaws.com'.

public_ipv4: str | None

The public IPv4 address of the instance, or None if the instance is not public, e.g. '1.2.3.4'.

region: str

The region the instance is running in, e.g. 'eu-west-1'.

reservation_id: str

The ID of the reservation used to launch the instance, e.g. 'r-12345678901234567'.

security_groups : list[str]

List of security groups by name, e.g. ['ssh-access', 'custom-sg-1'].

spot_instance_action: SpotInstanceAction | None

Uncached. An object describing an action about to happen to this spot instance. Returns None if the instance is not spot, or not marked for termination.

The SpotInstanceAction object has two attributes:

  • action: str - the action about to happen, one of "hibernate", "stop", or "terminate".
  • time: datetime - the approximate UTC datetime when the action will occur.

See AWS docs section for a little more information.

tags: InstanceTags

A dict-like mapping of the tags for the instance (documented below). This requires you to explicitly enable the feature for the instance. If the feature is not enabled, accessing this attribute raises an error.

(It also seems that there is a bug where if the feature is enabled and then disabled, the metadata service returns an empty response. This is indistinguishable from โ€œno tagsโ€, so beware that in that case, InstanceTags will just look like an empty mapping.)

user_data: bytes | None

The raw user data assigned to the instance (not base64 encoded), or None if there is none.

InstanceTags

A dict-like mapping of tag names to values (both strs). To avoid unnecessary requests, the mapping is lazy: values are only fetched when required. (Names are known on construction though, from the first request in EC2Metadata.tags.)

The metadata service will receive tag updates on some instance types, as per the AWS documentation:

If you add or remove an instance tag, the instance metadata is updated while the instance is running for instances built on the Nitro System, without needing to stop and then start the instance. For all other instances, to update the tags in the instance metadata, you must stop and then start the instance.

Because InstanceTags is cached, it wonโ€™t reflect such updates on Nitro instances unless you clear it first:

del ec2_metadata.tags
ec2_metadata.tags["Name"]  # fresh

NetworkInterface

Represents a single network interface, as retrieved from EC2Metadata.network_interfaces. Again like EC2Metadata all its attributes cache on first access, and can be cleared with del or its clear_all() method.

device_number: int

The unique device number associated with that interface, e.g. 0.

interface_id: str

The unique id used to identify the Elastic Network Interface, e.g. 'eni-12345'.

ipv4_associations: dict[str, list[str]]

A dictionary mapping public IP addresses on the interface to the list of private IP addresses associated with that public IP, for each public IP that is associated with the interface, e.g. {'54.0.0.1': ['172.30.0.0']}.

ipv6s: list[str]

The IPv6 addresses associated with the interface, e.g. ['2001:db8:abcd:ef00::1234'].

mac: str

The MAC address of the interface, e.g. '01:23:45:67:89:ab'.

owner_id: str

The AWS Account ID of the owner of the network interface, e.g. '123456789012'.

private_hostname: str

The interface's local/private hostname, e.g. 'ip-172-30-0-0.eu-west-1.compute.internal'.

private_ipv4s: list[str]

The private IPv4 addresses associated with the interface, e.g. ['172.30.0.0'].

public_hostname: str | None

The interface's public DNS (IPv4), e.g. 'ec2-54-0-0-0.compute-1.amazonaws.com'.

public_ipv4s: list[str]

The Elastic IP addresses associated with the interface, e.g. ['54.0.0.0'].

security_groups: list[str]

The names of the security groups to which the network interface belongs, e.g. ['ssh-access', 'custom-sg-1'].

security_group_ids: list[str]

The names of the security groups to which the network interface belongs, e.g. ['sg-12345678', 'sg-12345679'].

subnet_id: str

The ID of the subnet in which the interface resides, e.g. 'subnet-12345678'.

subnet_ipv4_cidr_block: str | None

The IPv4 CIDR block of the subnet in which the interface resides, or None if there is none, e.g. '172.30.0.0/24'.

subnet_ipv6_cidr_blocks: list[str]

The list of IPv6 CIDR blocks of the subnet in which the interface resides, e.g. ['2001:db8:abcd:ef00::/64']. If the subnet does not have any IPv6 CIDR blocks or the instance isn't in a VPC, the list will be empty, e.g. [].

vpc_id: str

The ID of the VPC in which the interface resides, e.g. 'vpc-12345678'.

vpc_ipv4_cidr_block: str | None

The IPv4 CIDR block of the VPC, or None if the instance isn't in a VPC, e.g. '172.30.0.0/16'.

vpc_ipv4_cidr_blocks: list[str]

The list of IPv4 CIDR blocks e.g. ['172.30.0.0/16']. If the interface doesnโ€™t have any such CIDR blocks, the list will be empty.

vpc_ipv6_cidr_blocks: list[str]

The list of IPv6 CIDR blocks of the VPC in which the interface resides, e.g. ['2001:db8:abcd:ef00::/56']. If the VPC does not have any IPv6 CIDR blocks or the instance isn't in a VPC, the list will be empty, e.g. [].

More Repositories

1

django-cors-headers

Django app for handling the server headers required for Cross-Origin Resource Sharing (CORS)
Python
5,087
star
2

django-htmx

Extensions for using Django with htmx.
JavaScript
866
star
3

django-upgrade

Automatically upgrade your Django projects.
Python
641
star
4

django-mysql

๐Ÿฌ ๐Ÿด Extensions to Django for use with MySQL/MariaDB
Python
535
star
5

blacken-docs

Run `black` on python code blocks in documentation files
Python
513
star
6

time-machine

Travel through time in your tests.
Python
447
star
7

flake8-comprehensions

โ„๏ธ A flake8 plugin to help you write better list/set/dict comprehensions.
Python
446
star
8

django-perf-rec

Keep detailed records of the performance of your Django code.
Python
330
star
9

django-browser-reload

Automatically reload your browser in development.
Python
296
star
10

mac-ansible

๐Ÿ„ Configuring my mac with Ansible
Shell
170
star
11

patchy

โš“ Patch the inner source of python functions at runtime.
Python
163
star
12

apig-wsgi

Wrap a WSGI application in an AWS Lambda handler function for running on API Gateway or an ALB.
Python
146
star
13

django-linear-migrations

Ensure your migration history is linear.
Python
136
star
14

treepoem

Barcode rendering for Python supporting QRcode, Aztec, PDF417, I25, Code128, Code39 and many more types.
PostScript
115
star
15

django-rich

Extensions for using Rich with Django.
Python
90
star
16

django-permissions-policy

Set the draft security HTTP header Permissions-Policy (previously Feature-Policy) on your Django app.
Python
81
star
17

django-watchfiles

Use watchfiles in Djangoโ€™s autoreloader.
Python
81
star
18

django-read-only

Disable Django database writes.
Python
75
star
19

django-minify-html

Use minify-html, the extremely fast HTML + JS + CSS minifier, with Django.
Python
73
star
20

flake8-tidy-imports

โ„๏ธ A flake8 plugin that helps you write tidier imports.
Python
60
star
21

heroicons

Use heroicons in your Django and Jinja templates.
Python
58
star
22

SublimeFiglet

Add in ASCII text art from "figlet"
Python
46
star
23

lifelogger

๐Ÿ“… Track your life like a pro on Google Calendar via your terminal.
Python
40
star
24

pip-lock

Check for differences between requirements.txt files and your environment
Python
36
star
25

django-capture-on-commit-callbacks

Capture and make assertions on transaction.on_commit() callbacks.
Python
35
star
26

django-version-checks

System checks for your project's environment.
Python
34
star
27

django-jsonfield

(Maintenance mode only) Cross-database JSON field for Django models.
Python
30
star
28

unittest-parametrize

Parametrize tests within unittest TestCases.
Python
29
star
29

scripts

Useful little scripts that I use on commandline. Work in OS-X + zsh at least.
Shell
27
star
30

multilint

โœ… Run multiple python linters easily
Python
27
star
31

flake8-no-pep420

A flake8 plugin to ban PEP-420 implicit namespace packages.
Python
22
star
32

django-startproject-templates

Python
22
star
33

pytest-is-running

pytest plugin providing a function to check if pytest is running.
Python
21
star
34

SublimeHTMLMustache

โœ๏ธ Adds HTML Mustache as a language to Sublime Text 2/3, with snippets.
19
star
35

pytest-reverse

Pytest plugin to reverse test order.
Python
19
star
36

owela-club

Play the Namibian game of Owela against a terrible AI. Built using Django and htmx.
Python
18
star
37

nose-randomly

๐Ÿ‘ƒ Nose plugin to randomly order tests and control `random.seed`
Python
17
star
38

talk-how-to-hack-a-django-website

JavaScript
14
star
39

dynamodb_utils

A toolchain for Amazon's DynamoDB to make common operations (backup, restore backups) easier.
Python
12
star
40

sound-resynthesis

๐Ÿ”ˆ Sound Resynthesis with a Genetic Algorithm - my final year project from university
Java
12
star
41

mariadb-dyncol

๐Ÿ’พ Python dicts <-> MariaDB Dynamic Column binary format
Python
11
star
42

pre-commit-oxipng

Mirror of oxipng for pre-commit.
Rust
11
star
43

pytest-flake8dir

โ„๏ธ A pytest fixture for testing flake8 plugins.
Python
11
star
44

logentries-cli

๐Ÿ“’ Get your logs from Logentries on the comandline.
Python
10
star
45

pre-commit-dprint

Mirror of dprint for pre-commit.
9
star
46

sublime-rst-improved

Python
8
star
47

h

Python
8
star
48

talk-improve-startup-time

โ€œHow to profile and improve startup timeโ€ talk
JavaScript
8
star
49

sublime_text_settings

โœ๏ธ My settings for sublime text 3 - as in Packages/User
Python
8
star
50

talk-django-and-htmx

JavaScript
7
star
51

django-settings-file

Python
7
star
52

tox-py

Adds the --py flag to tox to run environments matching a given Python interpreter.
Python
6
star
53

kwargs-only

A decorator to make a function accept keyword arguments only, on both Python 2 and 3.
Python
6
star
54

pytest-super-check

๐Ÿ”’ Pytest plugin to ensure all your TestCase classes call super() in setUp, tearDown, etc.
Python
6
star
55

django_atomic_celery

Atomic transaction aware Celery tasks for Django
Python
6
star
56

django-coverage-example

Python
5
star
57

django-pymysql-backend

A Django database backend for MySQL using PyMySQL.
Python
5
star
58

pytest-restrict

๐Ÿ”’ Pytest plugin to restrict the test types allowed
Python
5
star
59

talk-data-oriented-django

JavaScript
4
star
60

talk-speed-up-your-tests-with-setuptestdata

JavaScript
4
star
61

talk-django-and-web-security-headers

JavaScript
3
star
62

fluentd.tmLanguage

Syntax highlighting for Fluentd configuration files
3
star
63

django-ticket-33153

https://code.djangoproject.com/ticket/33153
Python
3
star
64

pytest-flake8-path

A pytest fixture for testing flake8 plugins.
Python
3
star
65

django_atomic_signals

Signals for atomic transaction blocks in Django 1.6+
Python
3
star
66

flake8-no-types

A flake8 plugin to ban type hints.
Python
3
star
67

SublimeMoveTabs

โœ๏ธ A short plugin for Sublime Text 2 that allows rearrangement of tabs/'views' with the keyboard.
Python
3
star
68

dynamodb_local_utils

Automatically run DynamoDB Local on Mac OS X
Shell
3
star
69

pygments-git

Pygments lexers for Git output and files
Python
3
star
70

workshop-evenergy-concurrency-and-parallelism

Python
2
star
71

talk-how-complex-systems-fail

Talk for the Papers We Love London meetup
TeX
2
star
72

google_lifelog

Making a lifelog on google calendar.
Python
2
star
73

talk-building-interactive-pages-with-htmx

JavaScript
2
star
74

workshop-idiomatic-python

Python
2
star
75

ansible-talk-custom-template-filters

My talk for the Ansible London Meetup in March 2015
TeX
2
star
76

talk-django-vs-flask

JavaScript
2
star
77

django-talk-factory-boy

Talk for London Django Meetup
TeX
2
star
78

ProgrammingInterview

Solving the problems posted on ProgrammingInterview on YouTube
Python
2
star
79

example-pre-commit-ci-lite

example
2
star
80

django-server-push-demo

Python
2
star
81

talk-django-capture-on-commit-callbacks

JavaScript
2
star
82

techblog

Filled with little coding notes and fixes.
2
star
83

adamchainz

๐Ÿ‘‹
Python
2
star
84

SublimeCowsay

โœ๏ธ๐Ÿฎ A silly little Sublime Text plugin for 2 and 3 to allow you to quickly convert a text selection to a cow speech bubble via the brilliant cowsay utility.
Python
2
star
85

djceu2019-workshop

Python
2
star
86

talk-what-happens-when-you-run-manage.py-test

JavaScript
2
star
87

talk-technologies-that-will-be-around-in-21-years

JavaScript
2
star
88

django-demo-constraint-single-column-not-null

Python
2
star
89

django-harlequin

Launch Harlequin, the SQL IDE for your Terminal, with your Django database configuration.
Python
2
star
90

channels-bug-connection-closed

Reproduction for Channels bug
Python
1
star
91

workshop-concurrency-and-parallelism

Python
1
star
92

django-talk-duth

Django Under The Hood 2015 Summary
TeX
1
star
93

workshop-rest-api-django

Python
1
star
94

workshop-recommended-practices

Python
1
star
95

talk-django-3.2-test-features

JavaScript
1
star
96

workshop-profiling-and-debugging

Python
1
star
97

django-feature-policy-shim

1
star
98

kvkit

high-level python toolkit for ordered key/value stores
Python
1
star
99

phabricator-csv-import

Python
1
star
100

django-blue-green-example

Reproducing the technique from โ€œSmooth Database Changes in Blue-Green Deploymentsโ€ by Mariusz Felisiak.
Python
1
star