• Stars
    star
    1,032
  • Rank 44,644 (Top 0.9 %)
  • Language
    Python
  • License
    BSD 3-Clause "New...
  • Created about 8 years ago
  • Updated 7 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Asyncio cache manager for redis, memcached and memory

aiocache

Asyncio cache supporting multiple backends (memory, redis and memcached).

https://travis-ci.org/argaen/aiocache.svg?branch=master https://api.codacy.com/project/badge/Grade/96f772e38e63489ca884dbaf6e9fb7fd

This library aims for simplicity over specialization. All caches contain the same minimum interface which consists on the following functions:

  • add: Only adds key/value if key does not exist.
  • get: Retrieve value identified by key.
  • set: Sets key/value.
  • multi_get: Retrieves multiple key/values.
  • multi_set: Sets multiple key/values.
  • exists: Returns True if key exists False otherwise.
  • increment: Increment the value stored in the given key.
  • delete: Deletes key and returns number of deleted items.
  • clear: Clears the items stored.
  • raw: Executes the specified command using the underlying client.

Installing

  • pip install aiocache
  • pip install aiocache[redis]
  • pip install aiocache[memcached]
  • pip install aiocache[redis,memcached]
  • pip install aiocache[msgpack]

Usage

Using a cache is as simple as

>>> import asyncio
>>> from aiocache import Cache
>>> cache = Cache(Cache.MEMORY) # Here you can also use Cache.REDIS and Cache.MEMCACHED, default is Cache.MEMORY
>>> with asyncio.Runner() as runner:
>>>     runner.run(cache.set('key', 'value'))
True
>>>     runner.run(cache.get('key'))
'value'

Or as a decorator

import asyncio

from collections import namedtuple

from aiocache import cached, Cache
from aiocache.serializers import PickleSerializer
# With this we can store python objects in backends like Redis!

Result = namedtuple('Result', "content, status")


@cached(
    ttl=10, cache=Cache.REDIS, key="key", serializer=PickleSerializer(), port=6379, namespace="main")
async def cached_call():
    print("Sleeping for three seconds zzzz.....")
    await asyncio.sleep(3)
    return Result("content", 200)


async def run():
    await cached_call()
    await cached_call()
    await cached_call()
    cache = Cache(Cache.REDIS, endpoint="127.0.0.1", port=6379, namespace="main")
    await cache.delete("key")

if __name__ == "__main__":
    asyncio.run(run())

The recommended approach to instantiate a new cache is using the Cache constructor. However you can also instantiate directly using aiocache.RedisCache, aiocache.SimpleMemoryCache or aiocache.MemcachedCache.

You can also setup cache aliases so its easy to reuse configurations

import asyncio

from aiocache import caches

# You can use either classes or strings for referencing classes
caches.set_config({
    'default': {
        'cache': "aiocache.SimpleMemoryCache",
        'serializer': {
            'class': "aiocache.serializers.StringSerializer"
        }
    },
    'redis_alt': {
        'cache': "aiocache.RedisCache",
        'endpoint': "127.0.0.1",
        'port': 6379,
        'timeout': 1,
        'serializer': {
            'class': "aiocache.serializers.PickleSerializer"
        },
        'plugins': [
            {'class': "aiocache.plugins.HitMissRatioPlugin"},
            {'class': "aiocache.plugins.TimingPlugin"}
        ]
    }
})


async def default_cache():
    cache = caches.get('default')   # This always returns the SAME instance
    await cache.set("key", "value")
    assert await cache.get("key") == "value"


async def alt_cache():
    cache = caches.create('redis_alt')   # This creates a NEW instance on every call
    await cache.set("key", "value")
    assert await cache.get("key") == "value"


async def test_alias():
    await default_cache()
    await alt_cache()

    await caches.get("redis_alt").delete("key")


if __name__ == "__main__":
    asyncio.run(test_alias())

How does it work

Aiocache provides 3 main entities:

  • backends: Allow you specify which backend you want to use for your cache. Currently supporting: SimpleMemoryCache, RedisCache using redis and MemCache using aiomcache.
  • serializers: Serialize and deserialize the data between your code and the backends. This allows you to save any Python object into your cache. Currently supporting: StringSerializer, PickleSerializer, JsonSerializer, and MsgPackSerializer. But you can also build custom ones.
  • plugins: Implement a hooks system that allows to execute extra behavior before and after of each command.
If you are missing an implementation of backend, serializer or plugin you think it could be interesting for the package, do not hesitate to open a new issue.

docs/images/architecture.png

Those 3 entities combine during some of the cache operations to apply the desired command (backend), data transformation (serializer) and pre/post hooks (plugins). To have a better vision of what happens, here you can check how set function works in aiocache:

docs/images/set_operation_flow.png

Amazing examples

In examples folder you can check different use cases:

Documentation

More Repositories

1

aiohttp

Asynchronous HTTP client/server framework for asyncio and Python
Python
14,457
star
2

aiomysql

aiomysql is a library for accessing a MySQL database from the asyncio
Python
1,748
star
3

aiopg

aiopg is a library for accessing a PostgreSQL database from the asyncio
Python
1,369
star
4

yarl

Yet another URL library
Python
1,200
star
5

aiobotocore

asyncio support for botocore library using aiohttp
Python
1,193
star
6

aiokafka

asyncio client for kafka
Python
1,143
star
7

aiojobs

Jobs scheduler for managing background task (asyncio)
Python
839
star
8

janus

Thread-safe asyncio-aware queue for Python
Python
774
star
9

async-lru

Simple LRU cache for asyncio
Python
710
star
10

aiohttp-demos

Demos for aiohttp project
Makefile
705
star
11

aiomonitor

aiomonitor is module that adds monitor and python REPL capabilities for asyncio application
Python
644
star
12

async-timeout

asyncio-compatible timeout class
Python
536
star
13

aiodocker

Python Docker API client based on asyncio and aiohttp
Python
429
star
14

aiozmq

Asyncio (pep 3156) integration with ZeroMQ
Python
420
star
15

multidict

The multidict implementation
Python
386
star
16

aiosmtpd

A reimplementation of the Python stdlib smtpd.py based on asyncio.
Python
319
star
17

create-aio-app

The boilerplate for aiohttp. Quick setup for your asynchronous web service.
Python
306
star
18

aioodbc

aioodbc - is a library for accessing a ODBC databases from the asyncio
Python
304
star
19

aiohttp-devtools

dev tools for aiohttp
Python
248
star
20

aiohttp-session

Web sessions for aiohttp.web
Python
238
star
21

aiohttp-security

auth and permissions for aiohttp
Python
229
star
22

aiohttp-jinja2

jinja2 template renderer for aiohttp.web
Python
228
star
23

aiohttp-admin

admin interface for aiohttp application http://aiohttp-admin.readthedocs.io
Python
216
star
24

aiohttp-cors

CORS support for aiohttp
Python
199
star
25

aiohttp-sse

Server-sent events support for aiohttp
Python
197
star
26

aiohttp-debugtoolbar

aiohttp_debugtoolbar is library for debugtoolbar support for aiohttp
JavaScript
193
star
27

aiozipkin

Distributed tracing instrumentation for asyncio with zipkin
Python
184
star
28

aioftp

ftp client/server for asyncio (http://aioftp.readthedocs.org)
Python
183
star
29

aiomcache

Minimal asyncio memcached client
Python
139
star
30

aioelasticsearch

aioelasticsearch-py wrapper for asyncio
Python
138
star
31

aiosignal

aiosignal: a list of registered asynchronous callbacks
Python
130
star
32

pytest-aiohttp

pytest plugin for aiohttp support
Python
129
star
33

aiorwlock

Read/Write Lock - synchronization primitive for asyncio
Python
129
star
34

sockjs

SockJS Server
Python
118
star
35

frozenlist

`FrozenList` is a `list`-like structure that implements `collections.abc.MutableSequence` and can be made immutable.
Python
95
star
36

aiocassandra

Simple threaded cassandra wrapper for asyncio
Python
88
star
37

aiohttp-remotes

A set of useful tools for aiohttp.web server
Python
79
star
38

aiologstash

asyncio logging handler for logstash
Python
58
star
39

aiocouchdb

CouchDB client built on top of aiohttp (asyncio)
Python
53
star
40

aioamqp_consumer

consumer/producer/rpc library built over aioamqp
Python
36
star
41

aiohttp-mako

mako template renderer for aiohttp.web
Python
31
star
42

aioneo4j

asyncio client for neo4j
Python
30
star
43

aiosparql

An asynchronous SPARQL client library using aiohttp
Python
26
star
44

sort-all

Sort __all__ lists alphabetically
Python
25
star
45

aioga

Google Analytics client for asyncio
Python
22
star
46

aioppspp

IETF PPSP RFC7574 in Python/asyncio
Python
21
star
47

sphinxcontrib-asyncio

Sphinx extension to add asyncio-specific markups
Python
20
star
48

aiohttp-benchmarks

Python
17
star
49

aioloop-proxy

A proxy for asyncio.AbstractEventLoop for testing purposes
Python
14
star
50

aiohttp-flashbag

The library provides flashbag for aiohttp
Python
10
star
51

idna-ssl

Patch ssl.match_hostname for Unicode(idna) domains support
Python
9
star
52

aiohttp-bot

A bot for automating boring tasks
Python
8
star
53

aio-libs.github.io

aio-libs static site
HTML
8
star
54

dynoname

Dynamic name resolution for asyncio libraries
Python
8
star
55

.github

Organization wide community settings
Python
7
star
56

triagers

A repo for request for aio-libs triage
5
star
57

get-releasenote

GitHub action for getting release note from towncrier rendered file
Python
5
star
58

aiohappyeyeballs

Happy Eyeballs for pre-resolved hosts
Python
5
star
59

create-release

GitHub action for release creation
4
star
60

aiohttp-site

A site for aiohttp project
HTML
3
star
61

prepare-coverage

Temporarily store coverage in Artifact storage
Makefile
3
star
62

aiohttp-theme

Sphinx theme for aiohttp
Python
2
star
63

azure-pipelines

Common azure pipelines templates used by aio-libs
2
star
64

night-watch

Ops purposes repo: validates Travis CI configs of all the repos in @aio-libs on daily basis
2
star
65

upload-coverage

Upload coverage chunks previously stored by prepare-coverage action
Makefile
1
star