• Stars
    star
    1,203
  • Rank 38,934 (Top 0.8 %)
  • Language
    Python
  • License
    MIT License
  • Created over 8 years ago
  • Updated about 1 month ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

simplified environment variable parsing

environs: simplified environment variable parsing

Latest version Build Status marshmallow 3 compatible Black code style

environs is a Python library for parsing environment variables. It allows you to store configuration separate from your code, as per The Twelve-Factor App methodology.

Contents

Features

  • Type-casting
  • Read .env files into os.environ (useful for local development)
  • Validation
  • Define custom parser behavior
  • Framework-agnostic, but integrates well with Flask and Django

Install

pip install environs

Basic usage

With some environment variables set...

export GITHUB_USER=sloria
export MAX_CONNECTIONS=100
export SHIP_DATE='1984-06-25'
export TTL=42
export ENABLE_LOGIN=true
export GITHUB_REPOS=webargs,konch,ped
export GITHUB_REPO_PRIORITY="webargs=2,konch=3"
export COORDINATES=23.3,50.0
export LOG_LEVEL=DEBUG

Parse them with environs...

from environs import Env

env = Env()
env.read_env()  # read .env file, if it exists
# required variables
gh_user = env("GITHUB_USER")  # => 'sloria'
secret = env("SECRET")  # => raises error if not set

# casting
max_connections = env.int("MAX_CONNECTIONS")  # => 100
ship_date = env.date("SHIP_DATE")  # => datetime.date(1984, 6, 25)
ttl = env.timedelta("TTL")  # => datetime.timedelta(0, 42)
log_level = env.log_level("LOG_LEVEL")  # => logging.DEBUG

# providing a default value
enable_login = env.bool("ENABLE_LOGIN", False)  # => True
enable_feature_x = env.bool("ENABLE_FEATURE_X", False)  # => False

# parsing lists
gh_repos = env.list("GITHUB_REPOS")  # => ['webargs', 'konch', 'ped']
coords = env.list("COORDINATES", subcast=float)  # => [23.3, 50.0]

# parsing dicts
gh_repos_priorities = env.dict(
    "GITHUB_REPO_PRIORITY", subcast_values=int
)  # => {'webargs': 2, 'konch': 3}

Supported types

The following are all type-casting methods of Env:

  • env.str
  • env.bool
  • env.int
  • env.float
  • env.decimal
  • env.list (accepts optional subcast and delimiter keyword arguments)
  • env.dict (accepts optional subcast_keys and subcast_values keyword arguments)
  • env.json
  • env.datetime
  • env.date
  • env.time
  • env.timedelta (assumes value is an integer in seconds)
  • env.url
  • env.uuid
  • env.log_level
  • env.path (casts to a pathlib.Path)
  • env.enum (casts to any given enum type specified in type keyword argument, accepts optional ignore_case keyword argument)

Reading .env files

# .env
DEBUG=true
PORT=4567

Call Env.read_env before parsing variables.

from environs import Env

env = Env()
# Read .env into os.environ
env.read_env()

env.bool("DEBUG")  # => True
env.int("PORT")  # => 4567

Reading a specific file

By default, Env.read_env will look for a .env file in current directory and (if no .env exists in the CWD) recurse upwards until a .env file is found.

You can also read a specific file:

from environs import Env

with open(".env.test", "w") as fobj:
    fobj.write("A=foo\n")
    fobj.write("B=123\n")

env = Env()
env.read_env(".env.test", recurse=False)

assert env("A") == "foo"
assert env.int("B") == 123

Handling prefixes

# export MYAPP_HOST=lolcathost
# export MYAPP_PORT=3000

with env.prefixed("MYAPP_"):
    host = env("HOST", "localhost")  # => 'lolcathost'
    port = env.int("PORT", 5000)  # => 3000

# nested prefixes are also supported:

# export MYAPP_DB_HOST=lolcathost
# export MYAPP_DB_PORT=10101

with env.prefixed("MYAPP_"):
    with env.prefixed("DB_"):
        db_host = env("HOST", "lolcathost")
        db_port = env.int("PORT", 10101)

Variable expansion

# export CONNECTION_URL=https://${USER:-sloria}:${PASSWORD}@${HOST:-localhost}/
# export PASSWORD=secret
# export YEAR=${CURRENT_YEAR:-2020}

from environs import Env

env = Env(expand_vars=True)

connection_url = env("CONNECTION_URL")  # =>'https://sloria:secret@localhost'
year = env.int("YEAR")  # =>2020

Validation

# export TTL=-2
# export NODE_ENV='invalid'
# export EMAIL='^_^'

from environs import Env
from marshmallow.validate import OneOf, Length, Email

env = Env()

# simple validator
env.int("TTL", validate=lambda n: n > 0)
# => Environment variable "TTL" invalid: ['Invalid value.']


# using marshmallow validators
env.str(
    "NODE_ENV",
    validate=OneOf(
        ["production", "development"], error="NODE_ENV must be one of: {choices}"
    ),
)
# => Environment variable "NODE_ENV" invalid: ['NODE_ENV must be one of: production, development']

# multiple validators
env.str("EMAIL", validate=[Length(min=4), Email()])
# => Environment variable "EMAIL" invalid: ['Shorter than minimum length 4.', 'Not a valid email address.']

Deferred validation

By default, a validation error is raised immediately upon calling a parser method for an invalid environment variable. To defer validation and raise an exception with the combined error messages for all invalid variables, pass eager=False to Env. Call env.seal() after all variables have been parsed.

# export TTL=-2
# export NODE_ENV='invalid'
# export EMAIL='^_^'

from environs import Env
from marshmallow.validate import OneOf, Email, Length, Range

env = Env(eager=False)

TTL = env.int("TTL", validate=Range(min=0, max=100))
NODE_ENV = env.str(
    "NODE_ENV",
    validate=OneOf(
        ["production", "development"], error="NODE_ENV must be one of: {choices}"
    ),
)
EMAIL = env.str("EMAIL", validate=[Length(min=4), Email()])

env.seal()
# environs.EnvValidationError: Environment variables invalid: {'TTL': ['Must be greater than or equal to 0 and less than or equal to 100.'], 'NODE_ENV': ['NODE_ENV must be one of: production, development'], 'EMAIL': ['Shorter than minimum length 4.', 'Not a valid email address.']}

env.seal() validates all parsed variables and prevents further parsing (calling a parser method will raise an error).

Serialization

# serialize to a dictionary of simple types (numbers and strings)
env.dump()
# {'COORDINATES': [23.3, 50.0],
# 'ENABLE_FEATURE_X': False,
# 'ENABLE_LOGIN': True,
# 'GITHUB_REPOS': ['webargs', 'konch', 'ped'],
# 'GITHUB_USER': 'sloria',
# 'MAX_CONNECTIONS': 100,
# 'MYAPP_HOST': 'lolcathost',
# 'MYAPP_PORT': 3000,
# 'SHIP_DATE': '1984-06-25',
# 'TTL': 42}

Defining custom parser behavior

# export DOMAIN='http://myapp.com'
# export COLOR=invalid

from furl import furl

# Register a new parser method for paths
@env.parser_for("furl")
def furl_parser(value):
    return furl(value)


domain = env.furl("DOMAIN")  # => furl('https://myapp.com')


# Custom parsers can take extra keyword arguments
@env.parser_for("choice")
def choice_parser(value, choices):
    if value not in choices:
        raise environs.EnvError("Invalid!")
    return value


color = env.choice("COLOR", choices=["black"])  # => raises EnvError

Usage with Flask

# myapp/settings.py

from environs import Env

env = Env()
env.read_env()

# Override in .env for local development
DEBUG = env.bool("FLASK_DEBUG", default=False)
# SECRET_KEY is required
SECRET_KEY = env.str("SECRET_KEY")

Load the configuration after you initialize your app.

# myapp/app.py

from flask import Flask

app = Flask(__name__)
app.config.from_object("myapp.settings")

For local development, use a .env file to override the default configuration.

# .env
DEBUG=true
SECRET_KEY="not so secret"

Note: Because environs depends on python-dotenv, the flask CLI will automatically read .env and .flaskenv files.

Usage with Django

environs includes a number of helpers for parsing connection URLs. To install environs with django support:

pip install environs[django]

Use env.dj_db_url, env.dj_cache_url and env.dj_email_url to parse the DATABASE_URL, CACHE_URL and EMAIL_URL environment variables, respectively.

For more details on URL patterns, see the following projects that environs is using for converting URLs.

Basic example:

# myproject/settings.py
from environs import Env

env = Env()
env.read_env()

# Override in .env for local development
DEBUG = env.bool("DEBUG", default=False)
# SECRET_KEY is required
SECRET_KEY = env.str("SECRET_KEY")

# Parse database URLs, e.g.  "postgres://localhost:5432/mydb"
DATABASES = {"default": env.dj_db_url("DATABASE_URL")}

# Parse email URLs, e.g. "smtp://"
email = env.dj_email_url("EMAIL_URL", default="smtp://")
EMAIL_HOST = email["EMAIL_HOST"]
EMAIL_PORT = email["EMAIL_PORT"]
EMAIL_HOST_PASSWORD = email["EMAIL_HOST_PASSWORD"]
EMAIL_HOST_USER = email["EMAIL_HOST_USER"]
EMAIL_USE_TLS = email["EMAIL_USE_TLS"]

# Parse cache URLS, e.g "redis://localhost:6379/0"
CACHES = {"default": env.dj_cache_url("CACHE_URL")}

For local development, use a .env file to override the default configuration.

# .env
DEBUG=true
SECRET_KEY="not so secret"

For a more complete example, see django_example.py in the examples/ directory.

Why...?

Why envvars?

See The 12-factor App section on configuration.

Why not os.environ?

While os.environ is enough for simple use cases, a typical application will need a way to manipulate and validate raw environment variables. environs abstracts common tasks for handling environment variables.

environs will help you

  • cast envvars to the correct type
  • specify required envvars
  • define default values
  • validate envvars
  • parse list and dict values
  • parse dates, datetimes, and timedeltas
  • parse expanded variables
  • serialize your configuration to JSON, YAML, etc.

Why another library?

There are many great Python libraries for parsing environment variables. In fact, most of the credit for environs' public API goes to the authors of envparse and django-environ.

environs aims to meet three additional goals:

  1. Make it easy to extend parsing behavior and develop plugins.
  2. Leverage the deserialization and validation functionality provided by a separate library (marshmallow).
  3. Clean up redundant API.

See this GitHub issue which details specific differences with envparse.

License

MIT licensed. See the LICENSE file for more details.

More Repositories

1

TextBlob

Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more.
Python
9,115
star
2

doitlive

Because sometimes you need to do it live
Python
3,434
star
3

konch

Configures your Python shell.
Python
406
star
4

PythonORMSleepy

Python ORM/ODM Examples, For The Sleepy
Python
212
star
5

pypi-cli

A command-line interface to the Python Package Index (PyPI). Get package info, download statistics, and more.
Python
168
star
6

dotfiles

sloria's dotfiles as Ansible roles
Shell
152
star
7

local-repl

🐚 Project-specific configuration for the Node.js REPL
JavaScript
152
star
8

flask-ghpages-example

An example of how you can deploy a static Flask app on Github pages
Python
113
star
9

textblob-aptagger

*Deprecated* A fast and accurate part-of-speech tagger for TextBlob.
Python
104
star
10

flask-konch

An improved shell command for the Flask CLI
Python
69
star
11

textblob-fr

French language support for TextBlob.
Python
58
star
12

sphinx-issues

A Sphinx extension for linking to your project's issue tracker
Python
53
star
13

sublime-html5-boilerplate

Sublime Text 2/3 snippet to generate HTML5 boilerplate
47
star
14

aiohttp-utils

Handy utilities for building aiohttp.web applications
Python
47
star
15

ped

πŸ‘ž Quickly open Python modules in your text editor
Python
43
star
16

webargs-starlette

Declarative request parsing and validation for Starlette with webargs
Python
39
star
17

gig

Generate .gitignore files from the command line or programmatically, in Python
Python
38
star
18

tbpaste

Sentiment analysis, as easy as copy-and-paste
Python
30
star
19

perspective-api-client

Node.js client for the Perspective API
JavaScript
28
star
20

AreYouSure.js

Inline confirmation dialogs in Javascript
JavaScript
26
star
21

cookiecutter-docopt

A Python command-line script template that uses docopt for arguments parsing
Python
24
star
22

webtest-plus

An extension of WebTest with useful extras, including requests-style authentication
Python
17
star
23

textfeel-web

An online sentiment analyzer built with Flask and TextBlob
JavaScript
15
star
24

designpatterns.py

Some design patterns, in Python flavor
Python
14
star
25

sir

(Work in progress) An open source assistant . Python 3 + aiohttp on the backend. ES6 + Redux + React on the frontend.
JavaScript
13
star
26

jrnl-render

Render a jrnl (jrnl.sh) file as a webpage.
TypeScript
13
star
27

psychopy-project-template

A template for a PsychoPy project. Aims to separate concerns of the experiment designer and the programmer.
Python
13
star
28

tinynetrc

Read and write .netrc files in Python
Python
13
star
29

vim-ped

πŸ‘Ÿ Quickly open Python modules in vim
Vim Script
10
star
30

usv

Tools for classifying and analyzing animal ultrasonic vocalizations
Python
10
star
31

flask-template

*ABANDONDED*. Use cookiecutter-flask instead.
JavaScript
9
star
32

webtest-aiohttp

Integration of WebTest with aiohttp.web applications
Python
9
star
33

sublime-selenium-snippets

Sublime Text 2/3 snippets for the Selenium WebDriver Python bindings
9
star
34

yopy

Zero characters communication for humans
Python
8
star
35

wtfhack

A silly website for finding open-source projects to work on inspired by WhatTheFuckShouldIMakeForDinner.com
JavaScript
6
star
36

jrnl-parse

Parses jrnl (jrnl.sh) files in Node.js or the browser.
JavaScript
6
star
37

webtest-asgi

Integration of WebTest with ASGI applications.
Python
6
star
38

sublime-nose-snippets

Sublime Text snippets for Python nose testing
5
star
39

python-subreddit-stats

Visualization of user traffic on python-related subreddits
CSS
5
star
40

device-inventory

A simple Django app for device asset management.
JavaScript
5
star
41

stimulus.py

Wrappers around PsychoPy stimuli that make it easy to present a study paradigm
Python
4
star
42

simplist

One-click collaborative lists
JavaScript
4
star
43

pipstat

DEPRECATED: Use pypi-cli instead: https://github.com/sloria/pypi-cli
Python
3
star
44

zettel-new-tab

Firefox/Chrome extension to display text files in new tabs
JavaScript
3
star
45

www

The Pelican site that generates stevenloria.com
CSS
3
star
46

samsung-prediction

Predicting user activity based on smartphone data
R
2
star
47

marshmallow-dashboard

Python
2
star
48

r-lcfd

A minimal R project skeleton
R
2
star
49

sublime-pytest-snippets

Sublime Text Snippets for pytest
2
star
50

pre-commit-docker

Run pre-commit (https://pre-commit.com) in Docker
Dockerfile
2
star
51

datasources

Python
1
star
52

scribnote

Python
1
star
53

reversr

It reverses audio. Nothing more.
JavaScript
1
star
54

cosroom

[For COS employees only] Find an open room in the COS office
JavaScript
1
star
55

sepal

JavaScript
1
star
56

read_env

reads .env files
Python
1
star