• Stars
    star
    1,088
  • Rank 40,859 (Top 0.9 %)
  • Language
    Elixir
  • License
    MIT License
  • Created about 7 years ago
  • Updated 8 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

In-memory and distributed caching toolkit for Elixir.

Nebulex 🌌

In-memory and distributed caching toolkit for Elixir.

CI Coverage Status Hex Version Docs License

Nebulex provides support for transparently adding caching into an existing Elixir application. Similar to Ecto, the caching abstraction allows consistent use of various caching solutions with minimal impact on the code.

Nebulex cache abstraction shields developers from directly dealing with the underlying caching implementations, such as Redis, Memcached, or even other Elixir cache implementations like Cachex. Additionally, it provides totally out-of-box features such as cache usage patterns, declarative annotation-based caching, and distributed cache topologies, among others.

See the getting started guide and the online documentation for more information.

Usage

You need to add nebulex as a dependency to your mix.exs file. However, in the case you want to use an external (a non built-in adapter) cache adapter, you also have to add the proper dependency to your mix.exs file.

The supported caches and their adapters are:

Cache Nebulex Adapter Dependency
Generational Local Cache Nebulex.Adapters.Local Built-In
Partitioned Nebulex.Adapters.Partitioned Built-In
Replicated Nebulex.Adapters.Replicated Built-In
Multilevel Nebulex.Adapters.Multilevel Built-In
Nil (special adapter that disables the cache) Nebulex.Adapters.Nil Built-In
Cachex Nebulex.Adapters.Cachex nebulex_adapters_cachex
Redis NebulexRedisAdapter nebulex_redis_adapter
Distributed with Horde Nebulex.Adapters.Horde nebulex_adapters_horde
Multilevel with cluster broadcasting NebulexLocalMultilevelAdapter nebulex_local_multilevel_adapter
Ecto Postgres table Nebulex.Adapters.Ecto nebulex_adapters_ecto

For example, if you want to use a built-in cache, add to your mix.exs file:

def deps do
  [
    {:nebulex, "~> 2.6"},
    {:shards, "~> 1.1"},     #=> When using :shards as backend
    {:decorator, "~> 1.4"},  #=> When using Caching Annotations
    {:telemetry, "~> 1.0"}   #=> When using the Telemetry events (Nebulex stats)
  ]
end

In order to give more flexibility and fetch only needed dependencies, Nebulex makes all dependencies optional. For example:

  • For intensive workloads, you may want to use :shards as the backend for the local adapter and having partitioned tables. In such a case, you have to add :shards to the dependency list.

  • For enabling the usage of declarative annotation-based caching via decorators, you have to add :decorator to the dependency list.

  • For enabling Telemetry events to be dispatched when using Nebulex, you have to add :telemetry to the dependency list. See telemetry guide.

  • If you want to use an external adapter (e.g: Cachex or Redis adapter), you have to add the adapter dependency too.

Then run mix deps.get in your shell to fetch the dependencies. If you want to use another cache adapter, just choose the proper dependency from the table above.

Finally, in the cache definition, you will need to specify the adapter: respective to the chosen dependency. For the local built-in cache it is:

defmodule MyApp.Cache do
  use Nebulex.Cache,
    otp_app: :my_app,
    adapter: Nebulex.Adapters.Local
end

Quickstart example

Assuming you are using Ecto and you want to use declarative caching:

# In the config/config.exs file
config :my_app, MyApp.PartitionedCache,
  primary: [
    gc_interval: :timer.hours(12),
    backend: :shards,
    partitions: 2
  ]

# Defining a Cache with a partitioned topology
defmodule MyApp.PartitionedCache do
  use Nebulex.Cache,
    otp_app: :my_app,
    adapter: Nebulex.Adapters.Partitioned,
    primary_storage_adapter: Nebulex.Adapters.Local
end

# Some Ecto schema
defmodule MyApp.Accounts.User do
  use Ecto.Schema

  schema "users" do
    field(:username, :string)
    field(:password, :string)
    field(:role, :string)
  end

  def changeset(user, attrs) do
    user
    |> cast(attrs, [:username, :password, :role])
    |> validate_required([:username, :password, :role])
  end
end

# The Accounts context
defmodule MyApp.Accounts do
  use Nebulex.Caching

  alias MyApp.Accounts.User
  alias MyApp.PartitionedCache, as: Cache
  alias MyApp.Repo

  @ttl :timer.hours(1)

  @decorate cacheable(cache: Cache, key: {User, id}, opts: [ttl: @ttl])
  def get_user!(id) do
    Repo.get!(User, id)
  end

  @decorate cacheable(cache: Cache, key: {User, username}, opts: [ttl: @ttl])
  def get_user_by_username(username) do
    Repo.get_by(User, [username: username])
  end

  @decorate cache_put(
              cache: Cache,
              keys: [{User, user.id}, {User, user.username}],
              match: &match_update/1,
              opts: [ttl: @ttl]
            )
  def update_user(%User{} = user, attrs) do
    user
    |> User.changeset(attrs)
    |> Repo.update()
  end

  @decorate cache_evict(
              cache: Cache,
              keys: [{User, user.id}, {User, user.username}]
            )
  def delete_user(%User{} = user) do
    Repo.delete(user)
  end

  def create_user(attrs \\ %{}) do
    %User{}
    |> User.changeset(attrs)
    |> Repo.insert()
  end

  defp match_update({:ok, value}), do: {true, value}
  defp match_update({:error, _}), do: false
end

See more Nebulex examples.

Important links

Testing

Testing by default spawns nodes internally for distributed tests. To run tests that do not require clustering, exclude the clustered tag:

$ mix test --exclude clustered

If you have issues running the clustered tests try running:

$ epmd -daemon

before running the tests.

Benchmarks

Nebulex provides a set of basic benchmark tests using the library benchee, and they are located within the directory benchmarks.

To run a benchmark test you have to run:

$ MIX_ENV=test mix run benchmarks/{BENCH_TEST_FILE}

Where BENCH_TEST_FILE can be any of:

  • local_with_ets_bench.exs: benchmark for the local adapter using :ets backend.
  • local_with_shards_bench.exs: benchmark for the local adapter using :shards backend.
  • partitioned_bench.exs: benchmark for the partitioned adapter.

For example, for running the benchmark for the local adapter using :shards backend:

$ MIX_ENV=test mix run benchmarks/local_with_shards_bench.exs

Additionally, you can also run performance tests using :basho_bench. See nebulex_bench example for more information.

Contributing

Contributions to Nebulex are very welcome and appreciated!

Use the issue tracker for bug reports or feature requests. Open a pull request when you are ready to contribute.

When submitting a pull request you should not update the CHANGELOG.md, and also make sure you test your changes thoroughly, include unit tests alongside new or changed code.

Before to submit a PR it is highly recommended to run mix check and ensure all checks run successfully.

Copyright and License

Copyright (c) 2017, Carlos Bolaños.

Nebulex source code is licensed under the MIT License.

More Repositories

1

shards

Partitioned ETS tables for Erlang and Elixir
Erlang
292
star
2

erlbus

Simple, Distributed and Scalable PubSub Message Bus written in Erlang
Erlang
233
star
3

kvx

Simple in-memory Key/Value Store written in Elixir using `cabol/ex_shards`
Elixir
99
star
4

west

WEST (Web/Event-driven Systems Tool) is another messaging tool written in Erlang, that enables the building of messaging-based systems
Erlang
43
star
5

ex_shards

Elixir wrapper for cabol/shards .
Elixir
39
star
6

nebulex_ecto

Nebulex and Ecto integration
Elixir
27
star
7

nebulex_examples

Nebulex Examples
Elixir
26
star
8

nebulex_redis_adapter

Nebulex adapter for Redis
Elixir
23
star
9

gen_buffer

A generic message buffer behaviour with pooling and back-pressure for Erlang/Elixir.
Erlang
16
star
10

jchash

Jump Consistent Hash NIF library for Erlang/Elixir
Erlang
15
star
11

cross_db

Simple and flexible database wrapper for Erlang
Erlang
10
star
12

oauth2_mnesia_backend

Mnesia backend for kivra/oauth2 project.
Erlang
7
star
13

dberl

NoSQL DB access tool written in Erlang.
Erlang
6
star
14

tcp_client

An Erlang TCP client connections manager
Erlang
5
star
15

nebulex_cluster

Cluster utilities for Nebulex adapters like Redis or Memcached
Elixir
4
star
16

erlang_adt

Abstract Data Types Examples In Erlang
Erlang
3
star
17

rps

Rock-Paper-Scissors Example using Phoenix Framework
Elixir
3
star
18

nebulex_adapters_cachex

A Nebulex adapter for Cachex
Elixir
3
star
19

oauth2_dberl_backend

dberl backend for kivra/oauth2 project.
Erlang
2
star
20

shards_bench

Performance tests for cabol/shards project.
Erlang
2
star
21

cross_db_couchbase

Couchbase adapter for cabol/cross_db
1
star
22

shards_kv

Simple in-memory KV store using cabol/shards.
1
star
23

nebulex_memcached_adapter

Nebulex adapter for Memcached
Elixir
1
star
24

phoenix_oauth2_example

OAuth2 example using Phoenix and kivra/oauth2
1
star
25

shards_pg

Erlang PG2 implementation using shards!
1
star
26

nebulex_adapters_partitioned

Nebulex adapter for partitioned cache topology
1
star
27

shards_dist

Distributed implementation for Shards
1
star
28

cabol.github.io

Blog
CSS
1
star
29

nebulex_mnesia_adapter

Nebulex adapter for Mnesia
1
star