Multi-Agent Reinforcement Learning papers
This is a collection of Multi-Agent Reinforcement Learning (MARL) papers. Each category is a potential start point for you to start your research. Some papers are listed more than once because they belong to multiple categories.
For MARL papers with code and MARL resources, please refer to MARL Papers with Code and MARL Resources Collection.
I will continually update this repository and I welcome suggestions. (missing important papers, missing categories, invalid links, etc.) This is only a first draft so far and I'll add more resources in the next few months.
This repository is not for commercial purposes.
My email: [email protected]
Overview
- Reviews
- Environments
- Dealing With Credit Assignment Issue
- Policy Gradient
- Communication
- Emergent
- Opponent Modeling
- Game Theoretic
- Hierarchical
- Ad Hoc Teamwork
- League Training
- Curriculum Learning
- Mean Field
- Transfer Learning
- Meta Learning
- Fairness
- Exploration
- Graph Neural Network
- Model-based
- NAS
- Safe Multi-Agent Reinforcement Learning
- From Single-Agent to Multi-Agent
- Discrete-Continuous Hybrid Action Spaces / Parameterized Action Space
- Role
- Diversity
- Sparse Reward
- Large Scale
- DTDE
- Decision Transformer
- Offline MARL
- Generalization
- Adversarial
- Multi-Agent Path Finding
- To be Categorized
- TODO
Reviews
Recent Reviews (Since 2019)
- A Survey and Critique of Multiagent Deep Reinforcement Learning
- An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective
- Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
- A Review of Cooperative Multi-Agent Deep Reinforcement Learning
- Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
- A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
- Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
- A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems
Other Reviews (Before 2019)
- If multi-agent learning is the answer, what is the question?
- Multiagent learning is not the answer. It is the question
- Is multiagent deep reinforcement learning the answer or the question? A brief survey Note that A Survey and Critique of Multiagent Deep Reinforcement Learning is an updated version of this paper with the same authors.
- Evolutionary Dynamics of Multi-Agent Learning: A Survey
- (Worth reading although they're not recent reviews.)
Environments
Dealing With Credit Assignment Issue
Value Decomposition
Other Methods
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| COMA：Counterfactual Multi-Agent Policy Gradients | https://github.com/oxwhirl/pymarl | AAAI | 2018 |
| LiCA：Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning | https://github.com/mzho7212/LICA | NIPS | 2020 |
Policy Gradient
Communication
Communication Without Bandwidth Constraint
Communication Under Limited Bandwidth
Emergent
Opponent Modeling
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| Bayesian Opponent Exploitation in Imperfect-Information Games | IEEE Conference on Computational Intelligence and Games | 2018 | |
| LOLA：Learning with Opponent-Learning Awareness | AAMAS | 2018 | |
| Variational Autoencoders for Opponent Modeling in Multi-Agent Systems | 2020 | ||
| Stable Opponent Shaping in Differentiable Games | 2018 | ||
| Opponent Modeling in Deep Reinforcement Learning | https://github.com/hhexiy/opponent | ICML | 2016 |
| Game Theory-Based Opponent Modeling in Large Imperfect-Information Games | AAMAS | 2011 | |
| Agent Modelling under Partial Observability for Deep Reinforcement Learning | NIPS | 2021 |
Game Theoretic
Hierarchical
Ad Hoc Teamwork
League Training
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| AlphaStar：Grandmaster level in StarCraft II using multi-agent reinforcement learning | Nature | 2019 |
Curriculum Learning
Mean Field
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| Mean Field Multi-Agent Reinforcement Learning | ICML | 2018 | |
| Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learning | The world wide web conference | 2019 | |
| Bayesian Multi-type Mean Field Multi-agent Imitation Learning | NIPS | 2020 |
Transfer Learning
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems | Journal of Artificial Intelligence Research | 2019 | |
| Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning | 2020 |
Meta Learning
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning | ICML | 2021 | |
| Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments | 2017 |
Fairness
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| FEN：Learning Fairness in Multi-Agent Systems | NIPS | 2019 | |
| Fairness in Multiagent Resource Allocation with Dynamic and Partial Observations | AAMAS | 2018 | |
| Fairness in Multi-agent Reinforcement Learning for Stock Trading | 2019 |
Exploration
Dense Reward Exploration
Sparse Reward Exploration
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| EITI/EDTI：Influence-Based Multi-Agent Exploration | https://github.com/TonghanWang/EITI-EDTI | ICLR | 2020 |
| Cooperative Exploration for Multi-Agent Deep Reinforcement Learning | ICML | 2021 | |
| Centralized Model and Exploration Policy for Multi-Agent | 2021 | ||
| REMAX: Relational Representation for Multi-Agent Exploration | AAMAS | 2022 |
Uncategorized
Graph Neural Network
Model-based
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| Model-based Multi-Agent Reinforcement Learning with Cooperative Prioritized Sweeping | 2020 |
NAS
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| MANAS: Multi-Agent Neural Architecture Search | 2019 |
Safe Multi-Agent Reinforcement Learning
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| MAMPS: Safe Multi-Agent Reinforcement Learning via Model Predictive Shielding | 2019 | ||
| Safer Deep RL with Shallow MCTS: A Case Study in Pommerman | 2019 |
From Single-Agent to Multi-Agent
Discrete-Continuous Hybrid Action Space / Parameterized Action Space
Role
Diversity
Sparse Reward
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems | NIPS | 2021 | |
| Individual Reward Assisted Multi-Agent Reinforcement Learning | https://github.com/MDrW/ICML2022-IRAT | ICML | 2022 |
Large Scale
DTDE
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| Networked Multi-Agent Reinforcement Learning in Continuous Spaces | IEEE conference on decision and control | 2018 | |
| Value Propagation for Decentralized Networked Deep Multi-agent Reinforcement Learning | NIPS | 2019 | |
| Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents | ICML | 2018 |
Decision Transformer
Offline MARL
Adversarial
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systems | 2022 | ||
| Distributed Multi-Agent Deep Reinforcement Learning for Robust Coordination against Noise | 2022 | ||
| On the Robustness of Cooperative Multi-Agent Reinforcement Learning | IEEE Security and Privacy Workshops | 2020 | |
| Towards Comprehensive Testing on the Robustness of Cooperative Multi-agent Reinforcement Learning | CVPR workshop | 2022 | |
| Robust Multi-Agent Reinforcement Learning via Minimax Deep Deterministic Policy Gradient | AAAI | 2019 | |
| Multi-agent Deep Reinforcement Learning with Extremely Noisy Observations | NIPS Deep Reinforcement Learning Workshop | 2018 | |
| Policy Regularization via Noisy Advantage Values for Cooperative Multi-agent Actor-Critic methods | 2021 |
Multi-Agent Path Finding
- TODO
To be Categorized
TODO
- Multi-Agent Path Finding
- Generalization in MARL
Citation
If you find this repository useful, please cite our repo:
@misc{chen2021multi,
author={Chen, Hao},
title={Multi-Agent Reinforcement Learning Papers},
year={2021}
publisher = {GitHub},
journal = {GitHub Repository},
howpublished = {\url{https://github.com/TimeBreaker/Multi-Agent-Reinforcement-Learning-papers}}
}