A comprehensive list of categorized reinforcement learning environments.
Started and maintained by Andrew Szot and Youngwoon Lee .
Two other resources for RL environments:
Environments are listed alphabetically.
Assistive-gym
6 assistive tasks (ScratchItch, BedBathing, Feeding, Drinking, Dressing, and ArmManipulation).
4 commercial robots (PR2, Jaco, Baxter, Sawyer).
2 human states: static or active (takes actions according to a separate control policy).
Customizable female and male human models. 40 actuated human joints (head, torso, arms, waist, and legs).Realistic human joint limit.
Dexterous Gym
Extensions of the OpenAI Gym Dexterous Manipulation Environments.
Multiple environments requiring cooperation between two hands (handing objects over, throwing/catching objects).
"Pen Spin" Environment - train a hand to spin a pen between its fingers.
DoorGym
Train a policy to open up various doors.
Unity integration.
Random door knob generator and door knob dataset.
Gym Gazebo 2
Toolkit for developing and comparing reinforcement learning algorithms using ROS 2 and Gazebo.
Gym Ignition
Provides the capability of creating reproducible robotics environments for reinforcement learning research.
Accelerated and multiprocess execution
IKEA Furniture Assembly
Complex long-horizon manipulation tasks.
Includes 80+ furniture models, customizable background, lighting
and textures.
Features Baxter, Sawyer, and more robots.
Meta-World
50 diverse robot manipulation tasks on a simulated Sawyer robotic arm.
Also includes a variety of evaluation modes varying the number of training and testing tasks.
Playroom
Variety of tasks in desk scenario.
Evaluation code and play dataset will be included soon.
RAISIM
Raisim is a physics engine for rigid-body dynamics simulation.
Although it is a general physics engine, it has been mainly
used/tested for robotics and reinforcement learning so far. It
features an efficient implementation of recursive algorithms for
articulated system dynamics (Recursive Newton-Euler and Composite
Rigid Body Algorithm). RaisimLib is an exported cmake package of
raisim.
RLBench
100 unique, hand designed tasks.
Vision-guided manipulation, imitation learning, multi-task
learning, geometric computer vision and few-shot learning.
Robosuite
A set of standard benchmarking tasks in robots.
Defines a framework for easily creating new tasks and environments.
Roboschool
Control robots in simulation.
Can use other physics engines other than MuJoCo.
Alternative to standard OpenAI Gym mujoco environments.
Easy to train multiple agents at once.
Rex-Gym
OpenAI Gym environments for an open-source quadruped robot (SpotMicro)
Bomberland
Multi-agent 2D grid environment based on Bomberman.
Coin-Run
Training environment which provides a metric for an agentโs
ability to transfer its experience to novel situations.
Gym Retro
Gym Retro lets you turn classic video games into Gym environments
for reinforcement learning and comes with integrations for ~1000.
games.
Holodeck
High Fidelity Simulator for Reinforcement Learning and Robotics Research.
MarLร : Reinforcement Learning + Minecraft
A high level API built on top of Project Malmร to facilitate Reinforcement Learning experiments with a great degree of generalizability, capable of solving problems in pseudo-random, procedurally changing single and multi agent environments within the world of the mediatic phenomenon game Minecraft.
Minecraft
Data API for the MineRLv0 dataset.
Also has minecraft environment simulator with basic built in tasks.
PHYRE
Benchmark for physical reasoning that contains a set of simple classical mechanics puzzles in a 2D enviroment.
Soccer Simulator
Can control one or all football players at a time.
Includes football academy for diverse scenarios such as various
passing scenarios.
StarCraft 2
Provides an interface for RL agents to interact with StarCraft 2,
getting observations and sending actions.
SuperMario
Gym wrapper for the Super Mario levels. Includes many levels.
TorchCraft
Python interface for playing "StarCraft: Brood War".
VizDoom
ViZDoom allows developing AI bots that play Doom using only the
visual information (the screen buffer).
Meta-World
50 diverse robot manipulation tasks on a simulated Sawyer robotic arm.
Also includes a variety of evaluation modes varying the number of training and testing tasks.
Multiworld
Variety of Gym GoalEnvs that return the goal in the observation.
Playroom
Variety of tasks in desk scenario.
Evaluation code and play dataset will be included soon.
RoboDesk
Multi-task RL benchmark that comes with tasks from easy to hard,
with dense and sparse rewards.
Based on the Playroom desk env, with more robust physics settings
and controls that are suitable for RL.
RLBench
100 unique, hand designed tasks.
Vision-guided manipulation, imitation learning, multi-task
learning, geometric computer vision and few-shot learning.
Cartpole Generalization
Test generalization through varying the mass and length of the pole
in CartPole.
Natural RL Environment
Play common gym tasks with randomly generated backgrounds to test
generalization.
DMControl Generalization Benchmark
Generalization benchmark for continuous control tasks from DeepMind Control Suite. Includes hundreds of environments with randomized colors and dynamic video backgrounds of varying difficulty.
Procgen
16 simple-to-use procedurally-generated environments which provide
a direct measure of how quickly a reinforcement learning agent
learns generalizable skills.
The environments run at high speed (thousands of steps per second)
on a single core.
Animal-AI Testbed
900 tasks reflecting various cognitive skills of animals.
Powered by Unity ml-agent.
Crafter
Open world survival game that evaluates many agent abilities within one environment.
Faster and easier than Minecraft but poses some of the same challenges.
Can be used to evaluate reward-based or unsupervised agents (e.g.
artificial curiosity).
DeepMind Lab
Provides a suite of challenging 3D navigation and puzzle-solving
tasks for learning agents.
gym-maze
A simple 2D maze environment where an agent (blue dot) finds its
way from the top left corner (blue square) to the goal at the
bottom right corner (red square).
The objective is to find the
shortest path from the start to the goal.
gym-minigrid
Lightweight and fast grid world implementation with various
included tasks.
Easily modifable and extendable.
gym-miniworld
Minimalistic 3D interior simulator as an alternative to VizDoom or
DMLab.
Easily modifable and extendable.
Obstacle Tower
Traverse through procedurally generated floors which get progressively harder.
Challenging visual inputs.
AI2THOR
An Interactive 3D Environment for Visual AI
Gibson
3d navigation in indoor scans
Habitat
AI Habitat enables training of embodied AI agents (virtual robots)
in a highly photorealistic & efficient 3D simulator, before
transferring the learned skills to reality
HoME: a Household Multimodal Environment
A platform for agents to learn from vision, audio, semantics, physics, and interaction with objects and other agents, all within a realistic context.
House3D
House3D is a virtual 3D environment which consists of thousands of
indoor scenes equipped with a diverse set of scene types, layouts
and objects sourced from the SUNCG dataset
It consists of over 45k indoor 3D scenes, ranging from studios to
two-storied houses with swimming pools and fitness rooms
All 3D objects are fully annotated with category labels
Multiple observation modalities
Fast rendering at thousands of frames per second
MINOS
MINOS is a simulator designed to support the development of
multisensory models for goal-directed navigation in complex indoor
environments.
MINOS leverages large datasets of complex 3D environments and
supports flexible configuration of multimodal sensor suites.
Nvidia ISAAC simulator
A virtual robotics laboratory and a high-fidelity 3D world simulator
VirtualHome
A 3D environment allowing to simulate and generate videos of activities as sequences of actions and interaction.
Massive Multi Agent Game Environment
We consider MMORPGs (Massive
Multiplayer Online Role Playing Games) the best proxy for the real
world among human games: they are complete macrocosms featuring
thousands of agents per persistent world, diverse skilling systems,
global economies, complex emergent social structures, and ad-hoc
high stakes single and team based conflict.
Multi-agent Particle Environment
A simple multi-agent particle world with a continuous observation and discrete action space, along with some basic simulated physics
OpenAI Multi-Agent Competition Environments
Contains many continous control, multi-agent tasks.
OpenAI Multi-Agent Hide and Seek
A team of seekers and a team of hiders.
Both teams can use tools to achieve their objective.
RoboSumo
Sumo-wrestling between two ants using continuous control.
SUMO-RL
Multi-agent traffic signal control using SUMO simulator.
Assistive-gym
6 assistive tasks (ScratchItch, BedBathing, Feeding, Drinking, Dressing, and ArmManipulation).
4 commercial robots (PR2, Jaco, Baxter, Sawyer).
2 human states: static or active (takes actions according to a separate control policy).
Customizable female and male human models. 40 actuated human joints (head, torso, arms, waist, and legs).Realistic human joint limit.
DeepMind AI Safety Gridworlds
This is a suite of reinforcement learning environments illustrating
various safety properties of intelligent agents.
Safety Gym
Tools for accelerating safe exploration research.
Autonomous Vehicle Simulator
Open source simulator for autonomous vehicles built on Unreal Engine
/ Unity, from Microsoft AI & Research
BARK-ML
Open source environments and reinforcement learning agents
for autonomous driving and behavior generation.
CARLA
CARLA has been developed from the ground up to support development,
training, and validation of autonomous driving systems
DeepDrive Self Driving Car Simulator
End-to-end simulation for self-driving cars
DeepMind StreetLearn
A C++/Python implementation of the StreetLearn environment based on
images from Street View, as well as a TensorFlow implementation of
goal-driven navigation agents solving the task published in โLearning
to Navigate in Cities Without a Mapโ, NeurIPS 2018
DeepGTAV v2
A plugin for GTAV that transforms it into a vision-based self-driving
car research environment.
DuckieTown
Self-driving car simulator for the Duckietown universe.
Highway-Env
A collection of environments for autonomous driving and tactical
decision-making tasks
SVL Simulator
Simulation software to accelerate safe autonomous vehicle development
Custom environment to support openai gym interface
TORCS
TORCS, The Open Racing Car Simulator is a highly portable multi
platform car racing simulation
Many tracks, opponents and cars available
Easy to modify
Full Body Muscle Simulator
A basic simulation and control for full-body Musculoskeletal system
Osim-rl
Reinforcement learning environments with musculoskeletal models. Task: learning to walk/move/run using musculoskeletal models.
Roboschool
Control robots in simulation.
Can use other physics engines other than MuJoCo.
Alternative to standard OpenAI Gym mujoco environments.
Easy to train multiple agents at once.
Jericho
A learning environment for man-made Interactive Fiction games.
TextWorld
TextWorld is a sandbox learning environment for the training
and evaluation of reinforcement learning (RL) agents on text-based
games.
Reco Gym
Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising.
RecSim
A Configurable Recommender Systems Simulation Platform from Google.
Gym-ANM
Environments that model Active Network Management (ANM) tasks in electricity distribution networks.
The list is not comprehensive, so please let us know if there is any environment that is missing, miscategorized, or needs a different description or image. Please submit an issue or open a pull request.
