MARL Resources Collection

This is a collection of Multi-Agent Reinforcement Learning (MARL) Resources. The purpose of this repository is to give beginners a better understanding of MARL and accelerate the learning process. Note that some of the resources are written in Chinese and only important papers that have a lot of citations were listed.

I will continually update this repository and I welcome suggestions. (missing important papers, missing important resources, invalid links, etc.) This is only a first draft so far and I'll add more resources in the next few months.

This repository is not for commercial purposes.

My email: [email protected]

Overview

• Courses
• Important Conferences
• Reviews
• Books
• Open Source Environments
• Research Groups
• Companies
• Paper List
• Talks
• Useful Resources
• TODO

Courses

• RLChina
• UCL Multi-agent AI
• SJTU Multi-Agent Reinforcement Learning Tutorial
• SJTU Reinforcement Learning

Important Conferences

• AAMAS, AAAI, IJCAI, ICLR, ICML, NIPS
• Sorted by difficulty (roughly)

Reviews

Recent Reviews (Since 2019)

• A Survey and Critique of Multiagent Deep Reinforcement Learning
• An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective
• Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
• A Review of Cooperative Multi-Agent Deep Reinforcement Learning
• Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
• A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

Other Reviews (Before 2019)

• If multi-agent learning is the answer, what is the question?
• Multiagent learning is not the answer. It is the question
• Is multiagent deep reinforcement learning the answer or the question? A brief survey Note that A Survey and Critique of Multiagent Deep Reinforcement Learning is an updated version of this paper with the same authors.
• Evolutionary Dynamics of Multi-Agent Learning: A Survey
• (Worth reading although they're not recent reviews.)

Books

• Multiagent systems: Algorithmic, game-theoretic, and logical foundations
• Multi‐Agent Machine Learning A Reinforcement Approach

Open Source Environments

• StarCraft Micromanagement Environment
  • pymarl is the original environment mentioned in the paper The StarCraft Multi-Agent Challenge. Note that pymarl is based on SMAC.
  • MARL-Algorithms is a simplified implementation of pymarl
  • EPyMARL is a extended python MARL framework with more environments (Level Based Foraging, Multi-Robot Warehouse, Multi-Agent Particle Environment) and more algorithms. Paper
  • pymarl2 added code-level tricks to the original pymarl. Paper
• Multi-Agent Particle Environment PyTorch Implementation
• Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents
• OpenSpiel: A Framework for Reinforcement Learning in Games
• Hanabi-learning-environment
• RoboCup 2D Half Field Offense
• Pommerman
• Multi-agent-emergence-environments
• Google Research Football
• MAgent Note that the original project is no longer maintained.
• DI-engine
• MARLlib is a MARL Extension for RLlib
• Multiagent Mujoco
• PettingZoo website
• Safe Policy Optimization (SafePO)
• (I personally recommend the first two environments for beginners, especially EPyMARL.)

Research Groups

Companies

• DeepMind
• OpenAI
• Facebook
• Tencent
• NetEase
• Huawei
• Parametrix.ai
• Inspir.ai

Paper Lists

• https://github.com/TimeBreaker/Multi-Agent-Reinforcement-Learning-papers
• https://github.com/TimeBreaker/MARL-papers-with-code
• https://github.com/LantaoYu/MARL-Papers

Talks

In English

• https://www.youtube.com/watch?v=W_9kcQmaWjo
• https://www.youtube.com/watch?v=TMTT2z8lifA
• https://www.youtube.com/watch?v=Yd6HNZnqjis
• https://www.youtube.com/watch?v=ufFue5_gR4c

In Chinese

• https://www.techbeat.net/talk-info?id=501
• https://www.bilibili.com/video/av457780236/
• https://space.bilibili.com/551888585/channel/detail?cid=167587
• https://www.bilibili.com/video/BV1ig4y1v7xU
• https://www.bilibili.com/video/BV18z411q7Kc
• https://www.bilibili.com/video/BV1k5411V7ue

Useful Resources

In English

• https://dblp.uni-trier.de/
• https://paperswithcode.com/
• https://www.connectedpapers.com
• https://deeplearn.org
• https://spinningup.openai.com/
• https://github.com/openai/spinningup
• https://github.com/Jinjiarui/hrl-papers

In Chinese

• http://www.neurondance.com/
• https://www.zhihu.com/question/376068768
• https://www.zhihu.com/question/323584412
• https://zhuanlan.zhihu.com/p/372558232
• https://space.bilibili.com/4801051?spm_id_from=333.788.b_765f7570696e666f.2
• https://www.zhihu.com/people/tian-yuan-dong
• https://www.zhihu.com/people/eyounx
• https://www.zhihu.com/people/wan-shang-zhu-ce-de
• Wechat public account: AIORHHC; RLCN
• https://www.bilibili.com/video/av925922430/
• https://www.bilibili.com/video/av626777400/
• https://github.com/NeuronDance/DeepRL

TODO

• The Research Groups part needs to be completed
• The Companies part needs to be completed
• The Useful Resources part needs to be perfected

Citation

If you find this repository useful, please cite our repo:

@misc{chen2021collection,
  author={Chen, Hao},
  title={A Collection of Multi-Agent Reinforcement Learning Resources},
  year={2021}
  publisher = {GitHub},
  journal = {GitHub Repository},
  howpublished = {\url{https://github.com/TimeBreaker/MARL-resources-collection}}
}