uber-research/ape-x uber-research/ape-x

  • Stars
    star
    188
  • Rank 198,863 (Top 5 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created almost 6 years ago
  • Updated about 5 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
uber-research/ape-x
github

uber-research

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

This repo replicates the results Horgan et al obtained in "Distributed Prioritized Experience Replay"

Replication of Ape-X (Distributed Prioritized Experience Replay)

This repo replicates the results Horgan et al obtained:

[1] Distributed Prioritized Experience Replay

Our code is based off of code from OpenAI baselines. The original code and related paper from OpenAI can be found here. Their implementation of DQN was modified to use Tensorflow custom ops.

Although Ape-X was originally a distributed algorithm, this implementation was meant to maximize throughput on a single machine. It was optimized for 2 GPUs (data gathering + optimization) but could be modified to use only one. With 2 GPUs and 20~40 CPUs you should be able to achieve human median performance in about 2 hours.

How to run

clone repo

git clone https://github.com/uber-research/ape-x.git

create python3 virtual env

python3 -m venv env
. env/bin/activate

install requirements

pip install tensorflow-gpu gym

Follow the setup under gym_tensorflow/README.md and run ./make to compile the custom ops.

launch experiment

python apex.py --env video_pinball --num-timesteps 1000000000 --logdir=/tmp/agent

Monitor your results with tensorboard

tensorboard --logdir=/tmp/agent

visualize results

python demo.py --env video_pinball --logdir=/tmp/agent

More Repositories

1

deep-neuroevolution

Deep Neuroevolution
Python
1,616
star
2

PPLM

Plug and Play Language Model implementation. Allows to steer topic and attributes of GPT-2 models.
Python
1,102
star
3

UPSNet

UPSNet: A Unified Panoptic Segmentation Network
Python
639
star
4

go-explore

Code for Go-Explore: a New Approach for Hard-Exploration Problems
Python
547
star
5

PyTorch-NEAT

Python
526
star
6

LaneGCN

[ECCV2020 Oral] Learning Lane Graph Representations for Motion Forecasting
Python
476
star
7

sbnet

Sparse Blocks Networks
Python
430
star
8

differentiable-plasticity

Implementations of the algorithms described in Differentiable plasticity: training plastic networks with gradient descent, a research paper from Uber AI Labs.
Python
394
star
9

DeepPruner

DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch (ICCV 2019)
Python
343
star
10

parallax

Tool for interactive embeddings visualization
Python
270
star
11

learning-to-reweight-examples

Code for paper "Learning to Reweight Examples for Robust Deep Learning"
Python
269
star
12

jpeg2dct

C++
251
star
13

poet

Paired Open-Ended Trailblazer (POET) and Enhanced POET
Python
235
star
14

intrinsic-dimension

Jupyter Notebook
220
star
15

CoordConv

Python
208
star
16

atari-model-zoo

A binary release of trained deep reinforcement learning models trained in the Atari machine learning benchmark, and a software release that enables easy visualization and analysis of models, and comparison across training algorithms.
Jupyter Notebook
201
star
17

EvoGrad

Python
178
star
18

TuRBO

Python
159
star
19

safemutations

safemutations
C++
143
star
20

permute-quantize-finetune

Using ideas from product quantization for state-of-the-art neural network compression.
Python
143
star
21

deconstructing-lottery-tickets

Python
142
star
22

CRISP

Python
131
star
23

metropolis-hastings-gans

Python
112
star
24

GTN

Python
75
star
25

backpropamine

Train self-modifying neural networks with neuromodulated plasticity
Python
73
star
26

loss-change-allocation

Python
61
star
27

MARVIN

Uber's Multi-Agent Routing Value Iteration Network
Python
52
star
28

GOCC

Go
51
star
29

Synthetic-Petri-Dish

Python
42
star
30

RxThreadEffectChecker

Static checker for Rx Threading Effects, based on the Checker Framework
Java
35
star
31

Map-Elites-Evolutionary

Map-Elites based on Evolution Strategies
Python
29
star
32

D3G

Estimating Q(s,s') with Deep Deterministic Dynamics Gradients
Python
29
star
33

java-dependency-validator

Dependency validator detects runtime compatibility issues at build time
Java
23
star
34

vargp

Variational Auto-Regressive Gaussian Processes for Continual Learning
Python
20
star
35

normative-uncertainty

Python
15
star
36

Evolvability-ES

Python
14
star
37

brezel

Starlark
8
star
38

dispatch-optim

Constrainted based optimization
Python
8
star
39

ga-world-models

Python
7
star
40

FSDM

Code tor the SIGDIAL 2019 paper Flexibly-Structured Model for Task-Oriented Dialogues. It implements a deep learning end-to-end differentiable dialogue system model
Python
7
star
41

rl-controller-verification

Quadcopter Verification
Python
5
star
42

go-context-propagate

Go
4
star
43

last-diff-analyzer

A multi-language tool for checking semantic equivalence for code
Go
2
star
44

tailr

TAILR
Python
1
star
45

xplane-bazel-docker

Bazel Xplane
C++
1
star