• Stars
    star
    115
  • Rank 305,916 (Top 7 %)
  • Language
    Python
  • License
    MIT License
  • Created about 3 years ago
  • Updated 6 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A clean and robust Pytorch implementation of PPO on continuous action space.

PPO-Continuous-Pytorch

A clean and robust Pytorch implementation of PPO on continuous action space:

Pendulum LunarLanderContinuous

Other RL algorithms by Pytorch can be found here.

Dependencies

gymnasium==0.29.1
numpy==1.26.1
pytorch==2.1.0

python==3.11.5

How to use my code

Train from scratch

python main.py

where the default enviroment is 'Pendulum'.

Play with trained model

python main.py --EnvIdex 0 --render True --Loadmodel True --ModelIdex 100

which will render the 'Pendulum'.

Change Enviroment

If you want to train on different enviroments, just run

python main.py --EnvIdex 1

The --EnvIdex can be set to be 0~5, where

'--EnvIdex 0' for 'Pendulum-v1'  
'--EnvIdex 1' for 'LunarLanderContinuous-v2'  
'--EnvIdex 2' for 'Humanoid-v4'  
'--EnvIdex 3' for 'HalfCheetah-v4'  
'--EnvIdex 4' for 'BipedalWalker-v3'  
'--EnvIdex 5' for 'BipedalWalkerHardcore-v3' 

Note: if you want train on BipedalWalker, BipedalWalkerHardcore, or LunarLanderContinuous, you need to install box2d-py first. You can install box2d-py via:

pip install gymnasium[box2d]

if you want train on Humanoid or HalfCheetah, you need to install MuJoCo first. You can install MuJoCo via:

pip install mujoco
pip install gymnasium[mujoco]

Visualize the training curve

You can use the tensorboard to record anv visualize the training curve.

  • Installation (please make sure PyTorch is installed already):
pip install tensorboard
pip install packaging
  • Record (the training curves will be saved at '\runs'):
python main.py --write True
  • Visualization:
tensorboard --logdir runs

Hyperparameter Setting

For more details of Hyperparameter Setting, please check 'main.py'

References

Proximal Policy Optimization Algorithms
Emergence of Locomotion Behaviours in Rich Environments

Training Curves

avatar
All the experiments are trained with same hyperparameters (see main.py).

More Repositories

1

DRL-Pytorch

Clean, Robust, and Unified PyTorch implementation of popular Deep Reinforcement Learning (DRL) algorithms (Q-learning, Duel DDQN, PER, C51, Noisy DQN, PPO, DDPG, TD3, SAC, ASL)
Python
1,151
star
2

TD3-BipedalWalkerHardcore-v2

Solve BipedalWalkerHardcore-v2 with TD3
Python
79
star
3

PPO-Discrete-Pytorch

A clean and robust Pytorch implementation of PPO on Discrete action space
Python
56
star
4

SAC-Continuous-Pytorch

a clean and robust Pytorch implementation of SAC on continuous action space
Python
53
star
5

SAC-Discrete-Pytorch

A clean and robust Pytorch implementation of SAC on discrete action space
Python
30
star
6

Duel-Double-DQN-Pytorch

A clean and robust implementation of Duel Double DQN
Python
27
star
7

OkayPlan

OkayPlan: A real-time global path palnning algorithm for dynamic environments
Python
27
star
8

TD3-Pytorch

A clean and robust Pytorch implementation of TD3 on continuous action space
Python
20
star
9

Actor-Sharer-Learner

Actor-Sharer-Learner training framework for off-policy DRL algorithms
Python
19
star
10

Prioritized-Experience-Replay-DDQN-Pytorch

A clean and robust implementation of Prioritized DQN and Prioritized Double DQN
Python
17
star
11

Sparrow-V0

A Reinforcement Learning Friendly Simulator for Mobile Robot
Python
16
star
12

okayplan_ros

Real-time global path planning algorithm for dynamic environments
Python
13
star
13

DDPG-Pytorch

A clean Pytorch implementation of DDPG on continuous action space.
Python
12
star
14

Real-time-Path-planning-with-SEPSO

Efficient Real-time Path Planning with SEPSO in Dynamic Scenarios
Python
10
star
15

Color

Color: Train a Real-world Local Path Planner in One Hour via Partially Decoupled Reinforcement Learning and Vectorized Diversity
10
star
16

Noisy-Duel-DDQN-Atari-Pytorch

A clean and robust implementation of Noisy-Duel-DDQN on Atari games
Python
8
star
17

Sparrow-V1

A Reinforcement Learning Friendly Simulator for Mobile Robot
Python
7
star
18

Q-learning

An implementation of Q-learning
Python
5
star
19

C51-Categorical-DQN-Pytorch

A clean and robust Pytorch implementation of Categorical DQN (C51)
Python
2
star