instadeepai/Mava

A framework for distributed multi-agent reinforcement learning in JAX

Welcome to Mava! 🦁

Installation | Quickstart | Documentation

Mava is a library for building multi-agent reinforcement learning (MARL) systems. Mava provides useful components, abstractions, utilities and tools for MARL and allows for simple scaling for multi-process system training and execution while providing a high level of flexibility and composability. Originating in the Research Team at InstaDeep, Mava is now developed jointly with the open-source community. “Mava” means experience, or wisdom, in Xhosa - one of South Africa’s eleven official languages.

To join us in these efforts, please feel free to reach out, raise issues or read our contribution guidelines (or just star 🌟 to stay up to date with the latest developments)!

👋 UPDATE - 01/10/2022: In the next few weeks, we will release our first JAX system! Using a more modular and flexible build method, with many more developments to come soon. These will include more components and utilities, state-of-the-art system implementations, notebooks and tutorials as well as detailed benchmarking and performance guides. After this release, our TF2-based systems will be deprecated. To still use them please install v0.1.3 of Mava (e.g. pip install id-mava==0.1.3). Please note, we will no longer be supporting these systems after our next release (v0.1.4).

Overview 🦜

• 🥑 Modular building blocks for MARL: modular abstractions and components for MARL to facilitate building multi-agent systems at scale.
• 🍬 Environment Wrappers: easily connect to your favourite MARL environment including SMAC, PettingZoo, Flatland and more. For details on our environment wrappers and how to add your own environment, please see here.
• 🎓 Educational Material: examples and user guides to facilitate Mava's adoption and highlight the added value of JAX-based MARL.

Installation 🎬

You can install the latest release of Mava as follows:

pip install id-mava[reverb,jax,envs]

You can also install directly from source:

pip install "id-mava[reverb,jax,envs] @ git+https://github.com/instadeepai/mava.git"

We have tested mava on Python 3.8, 3.9 and 3.10. Note that because the installation of JAX differs depending on your hardware accelerator, we advise users to explicitly install the correct JAX version (see the official installation guide). For more in-depth instalations guides including Docker builds and virtual environments, please see our detailed installation guide.

Quickstart ⚡

We have a Quickstart notebook that can be used to quickly create and train your first Multi-Agent System. For more on Mava's implementation details, please visit our documentation.

Contributing 🤝

Please read our contributing docs for details on how to submit pull requests, our Contributor License Agreement and community guidelines.

Troubleshooting and FAQs

Please read our troubleshooting and FAQs guide.

Citing Mava

If you use Mava in your work, please cite the accompanying technical report (to be updated soon to reflect our transition to JAX):

@article{pretorius2021mava,
    title={Mava: A Research Framework for Distributed Multi-Agent Reinforcement Learning},
    author={Arnu Pretorius and Kale-ab Tessera and Andries P. Smit and Kevin Eloff
    and Claude Formanek and St John Grimbly and Siphelele Danisa and Lawrence Francis
    and Jonathan Shock and Herman Kamper and Willie Brink and Herman Engelbrecht
    and Alexandre Laterre and Karim Beguir},
    year={2021},
    journal={arXiv preprint arXiv:2107.01460},
    url={https://arxiv.org/pdf/2107.01460.pdf},
}