Atari - Deep Reinforcement Learning algorithms in TensorFlow

Learning to play Atari in TensorFlow using Deep Reinforcement Learning

Setup

git clone https://github.com/brendanator/atari-rl
git submodule update --init
conda create --name atari-rl python=3.5
source activate atari-rl
conda install -y -c https://conda.binstar.org/menpo opencv3
conda install -y h5py numpy
pip install tensorflow
pip install 'gym[atari]'

Python 2.7 is also supported

Usage

• Show all options - python main.py --help
• Play a specific Atari game - python main.py --game Breakout

Papers Implemented

• ✅ Human Level Control through Deep Reinforcement Learning
  • python main.py
• ✅ Deep Reinforcement Learning with Double Q-learning
  • python main.py --double_q
• ✅ Dueling Network Architectures for Deep Reinforcement Learning
  • python main.py --dueling
• ✅ Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening
  • python main.py --optimality_tightening
• ✅ Prioritized Experience Replay
  • python main.py --replay_prioritized
  • Only proportional prioritized replay is implemented
• ✅ Unifying Count-Based Exploration and Intrinsic Motivation
  • python main.py --exploration_bonus
• ✅ Deep Exploration via Bootstrapped DQN
  • python main.py --bootstrapped
• ✅ Increasing the Action Gap: New Operators for Reinforcement Learning
  • python main.py --persistent_advantage_learning
• ✅ Learning values across many orders of magnitudes
  • python main.py --reward_scaling
• ✅ Asynchronous Methods for Deep Reinforcement Learning
  • python main.py --async one_step
  • python main.py --async n_step
  • python main.py --async n_step --sarsa
  • python main.py --async a3c
• ❌ Deep Recurrent Q-Learning for Partially Observable MDPs
• ❌ Safe and efficient Off-Policy Reinforcement Learning
• ❌ Continuous Deep Q-Learning with Model-based Acceleration

Acknowledgements

• https://github.com/mgbellemare/SkipCTS - Used in implementation of Unifying Count-Based Exploration and Intrinsic Motivation
• https://github.com/Kaixhin/Atari
• https://github.com/carpedm20/deep-rl-tensorflow