• Stars
    star
    173
  • Rank 220,124 (Top 5 %)
  • Language
    Jupyter Notebook
  • License
    MIT License
  • Created over 6 years ago
  • Updated almost 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Learning human driver models from NGSIM data with imitation learning.

NGSIM Env

  • This is a rllab environment for learning human driver models with imitation learning

Description

  • This repository does not contain a gail / infogail / hgail implementation
    • The reason ngsim_env does not contain the GAIL algorithm implementation is to enable the codebase to be more modular. This design decision enables ngsim_env to be used as an environment in which any imitation learning algorithm can be tested. Similarly, this design decision enables the GAIL algorithm to be a separate module that can be tested in any environment be that ngsim_env or otherwise. The installation process below gets the GAIL implementation from sisl/hgail
  • It also does not contain the human driver data you need for the environment to work. The installation process below gets the data from sisl/NGSIM.jl.
  • Figure below shows a diagram of the repositories ngsim_env depends on: dependecies

Installation Process

Step-by-step install instructions are at docs/install_env_gail_full.md

Train and run a single agent GAIL policy:

  1. Navigate to ngsim_env/scripts/imitation
  2. Train a policy, this involves running imitate.py
python imitate.py --exp_name NGSIM-gail --n_itr 1000 --policy_recurrent True
  1. Run the trained policy by using it to drive a car (this creates trajectories on all NGSIM sections using the trained policy). The training step was called imitate. This step is called validate.
python validate.py --n_proc 5 --exp_dir ../../data/experiments/NGSIM-gail/ --params_filename itr_1000.npz --random_seed 42
  1. Visualize the results: Open up a jupyter notebook and use the visualize*.ipynb files.
  • the visualize family of ipynb's have headers at the top of each file describing what it does.
    • visualize.ipynb is for extracting the Root Mean Square Error
    • visualize_trajectories.ipynb creates videos such as the one shown below in the demo section
    • visualize_emergent.ipynb calculates the emergent metrics such as offroad duration and collision rate

Training process: details

How this works?

  • See README files individual directories for details, but a high-level description is:
  • The python code uses pyjulia to instantiate a Julia interpreter, see the python directory for details
  • The driving environment is then built in Julia, see the julia directory for details
  • Each time the environment is stepped forward, execution passes from python to julia, updating the environment

Demo

To reproduce our experiments for the multiagent gail paper submitted to IROS, see

scripts/imitation/README.md

GAIL in a single-agent environment

Single agent GAIL (top) and PS-GAIL (bottom) in a multi-agent environment

References


If you found this library useful in your research, please consider citing our paper and/or paper:

@inproceedings{bhattacharyya2018multi,
  title={Multi-agent imitation learning for driving simulation},
  author={Bhattacharyya, Raunak P and Phillips, Derek J and Wulfe, Blake and Morton, Jeremy and Kuefler, Alex and Kochenderfer, Mykel J},
  booktitle={2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  pages={1534--1539},
  year={2018},
  organization={IEEE}
}

@article{bhattacharyya2019simulating,
  title={Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learning},
  author={Bhattacharyya, Raunak P and Phillips, Derek J and Liu, Changliu and Gupta, Jayesh K and Driggs-Campbell, Katherine and Kochenderfer, Mykel J},
  journal={arXiv preprint arXiv:1903.05766},
  year={2019}
}

More Repositories

1

MADRL

Repo containing code for multi-agent deep reinforcement learning (MADRL).
Python
659
star
2

tufte_algorithms_book

A template for textbooks in the same style as Algorithms for Optimization
TeX
350
star
3

NeuralVerification.jl

Methods to soundly verify deep neural networks
Julia
222
star
4

BayesNets.jl

Bayesian Networks for Julia
Julia
217
star
5

aa228-notebook

IJulia notebooks for AA228/CS238 Decision Making Under Uncertainty course at Stanford University
Jupyter Notebook
135
star
6

gail-driver

Python
111
star
7

MPOPIS

Adaptive importance sampling modification to MPPI
Julia
76
star
8

MultiAgentAllocationTransit.jl

Efficient Large-Scale Multi-Drone Delivery Using Transit Networks
Julia
75
star
9

OptimizationCourse

Jupyter Notebook
74
star
10

Chimp

General purpose framework for deep reinforcement learning
Python
72
star
11

AutomotiveDrivingModels.jl

Driving simulation architecture for Julia
Julia
63
star
12

SatelliteDynamics.jl

SatelliteDynamics.jl is a satellite dynamics modeling package written for Julia.
Julia
53
star
13

GridInterpolations.jl

Multidimensional grid interpolation in arbitrary dimensions
Julia
52
star
14

AttentionAugmentedConvLSTM

Implementation of TAAConvLSTM and SAAConvLSTM used in "Attention Augmented ConvLSTM for Environment Prediction"
Python
52
star
15

SCoBA.jl

Stochastic Conflict-Based Allocation
Julia
46
star
16

deep_flow_control

Source code for "Deep Dynamical Modeling and Control of Unsteady Fluid Flows" from NeurIPS 2018
Python
44
star
17

Crux.jl

Julia library for deep reinforcement learning
Julia
44
star
18

GaussianFilters.jl

Julia Package for discrete-time linear Gaussian parametric filtering systems, namely KF, EKF, UKF, GM-PHD
Jupyter Notebook
44
star
19

PedestrianAvoidancePOMDP.jl

POMDP formulation of a pedestrian avoidance problem for autonomous driving
Jupyter Notebook
43
star
20

ExprOptimization.jl

Algorithms for optimization of Julia expressions
Jupyter Notebook
42
star
21

DICG

Deep Implicit Coordination Graphs
Python
41
star
22

NNet

Documentation and scripts related to the .nnet file format. This file format specifies a simple text file to define feed-forward, fully-connected, ReLU activated neural networks. Example networks in this format can be found in the Reluplex repository.
Python
40
star
23

variational_koopman

Source code for "Deep Variational Koopman Models: Inferring Koopman Observations for Uncertainty-Aware Dynamics Modeling and Control" from IJCAI 2019.
Python
37
star
24

NGSIM.jl

A Julia package for handling the Next Generation Simulation (NGSIM) traffic dataset
Jupyter Notebook
36
star
25

AdaptiveStressTestingToolbox

A toolbox for worst-case validation of autonomous policies
Python
35
star
26

AutoViz.jl

Provides visualization tools for AutomotiveDrivingModels. Built on Cairo
Julia
33
star
27

hgail

gail, infogail, hierarchical gail implementations
Python
32
star
28

juliaplots.sty

A latex package for displaying plots made in Julia
TeX
32
star
29

AutonomousMerging.jl

Implementation of a highway merging scenario
Julia
30
star
30

aircraftshapes

Aircraft shapes for use with the TikZ LaTeX package.
TeX
30
star
31

D3Trees.jl

Flexible interactive visualization for large trees in Julia using D3.js
Julia
29
star
32

PicoSAT.jl

Julia bindings to the SAT solver picosat
Julia
29
star
33

InteractionSimulator

Simulator for the INTERACTION dataset
Python
28
star
34

AutomotiveSimulator.jl

Package for automotive simulation in julia
Julia
27
star
35

InteractionImitation

Imitation Learning with the INTERACTION Dataset
Python
27
star
36

AA120Q

AA120Q Course Materials
Julia
26
star
37

AutomotiveSafeRL

Training and evaluation scripts for applying formal methods and reinforcement learning to autonomous driving problems.
Jupyter Notebook
25
star
38

ExprRules.jl

Functions for declaring and working with grammars and expression trees in Julia.
Julia
24
star
39

Multilane.jl

Simulation and control of an autonomous car on a multilane highway in a (PO)MDP framework
Julia
23
star
40

AutomotivePOMDPs.jl

Jupyter Notebook
23
star
41

mechamodlearn

Structured framework for learning mechanical systems in PyTorch
Python
23
star
42

SceneInformer

This is the official implementation of the paper "Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments" by Bernard Lange, Jiachen Li, and Mykel J. Kochenderfer.
Jupyter Notebook
22
star
43

HorizontalCAS

C
21
star
44

AA228-CS238-Student

Starter code and data files for AA228/CS238 at Stanford University, Autumn 2021
Julia
20
star
45

BetaZero.jl

Belief-state planning for POMDPs using learned approximations
Julia
20
star
46

LOPR

This is the official implementation of LOPR used in "LOPR: Latent Occupancy PRediction using Generative Models"
Python
19
star
47

CustomerSim

Customer simulation for direct marketing experiments
Python
19
star
48

pygments-julia

Pygments lexer for Julia
Python
18
star
49

MultiAgentVariationalOcclusionInference

Multi-agent occlusion inference using observed driver behaviors. A driver sensor model is learned using a conditional variational autoencoder. Information from multiple drivers is fused into an ego vehicle's map using evidential theory.
Python
18
star
50

Discretizers.jl

A Julia package for data discretization and label maps
Julia
18
star
51

OVERT.jl

Relational piecewise-linear overapproximations of multi-dimensional functions
Jupyter Notebook
17
star
52

Double-Prong-Occupancy

Double-Prong ConvLSTM for Spatiotemporal Occupancy Prediction in Dynamic Environments
Python
17
star
53

InterpretableSelfAwarePrediction

Python
17
star
54

DreamrHHP

Julia
16
star
55

vehicleshapes

Vehicle (car) shapes for use with the Tikz LaTeX package.
TeX
15
star
56

Spot.jl

Julia wrapper for the Spot LTL and automata manipulation library
Julia
15
star
57

terminal-airspace-models

Repository for "Learning Probabilistic Trajectory Models of Aircraft in Terminal Airspace from Position Data"
Jupyter Notebook
15
star
58

latent_driver

Jointly learning policies and latent representations for driver behavior.
Julia
15
star
59

PyroRL

An RL environment made for wildfire evacuation.
Python
14
star
60

DecNashPlanning

Python
14
star
61

XPlaneConnectX

Python and Julia code for interfacing with X-Plane through UDP; similarly to XPlaneConnect, but also works for X-Plane 12.
Julia
14
star
62

AutonomousRiskFramework.jl

Framework for autonomous vehicle risk assessment
Julia
14
star
63

POMDPModelChecking.jl

Perform Model Checking and POMDP Planning from LTL specifications using POMDPs.jl
Julia
14
star
64

AutomotiveVisualization.jl

Visualize automotive simulation from AutomotiveSimulator.jl
Julia
14
star
65

BOMCP.jl

Bayesian Optimal Monte Carlo Planning POMDP solver
Julia
14
star
66

MPHRL

Model Primitive Hierarchical Reinforcement Learning
Python
13
star
67

EmergencyBrakingSystem.jl

Autonomous emergency braking system for pedestrian avoidance
Julia
13
star
68

POMDPStressTesting.jl

Adaptive stress testing of black-box systems within POMDPs.jl
Julia
13
star
69

AdversarialDriving.jl

Adversarial driving simulator for testing safety validation algorithms
Julia
13
star
70

MineralExploration

Julia
13
star
71

Vec.jl

2D and 3D vectors and their operations for Julia
Julia
13
star
72

ROSOccupancyGridPrediction

C++
11
star
73

JuliaPackageTemplate.jl

Package template and guide for setting up continuous integration and documentation deployment.
Julia
11
star
74

action_suggestions

A method of collaborative decision making using action suggestions by using the agent's policy to estimate the distribution over suggestions and treating a suggested action as an observation of the environment to update the agent's belief.
Julia
11
star
75

ConstructionBots.jl

An open-source multi-robot manufacturing simulator designed to test algorithms for multi-robot assembly planning.
Julia
10
star
76

SOFA.jl

Unofficial wrapper of the IAU SOFA C libraries for fundamental astronomy.
Julia
10
star
77

VerticalCAS

Julia
10
star
78

GitHub-ForceLargeFiles

Forcefully push large files to GitHub without using LFS by automatically detecting, compressing and splitting β‰₯100 MB files into partitions.
Python
10
star
79

TaskGraphs.jl

Precedence constrained multi-agent task assignment and path finding
Julia
10
star
80

pomdpland

A tour of Pomdpland
TeX
10
star
81

EvSoftmax

Code for Paper "Evidential Softmax for Sparse MultimodalDistributions in Deep Generative Models"
Python
10
star
82

ConflictAvoidanceDASC

Conflict avoidance algorithm for unmanned aircraft traffic management
Julia
9
star
83

SignalTemporalLogic.jl

Signal temporal logic (STL) formulas and robustness in Julia
Julia
9
star
84

LSTM-acc-predict

Lua
9
star
85

aa222-notebook

Stanford's AA222 IJulia Notebooks
Jupyter Notebook
9
star
86

StateEstimation.jl

Belief state estimation for Stanford's CS238/AA228 Decision Making Under Uncertainty
Julia
9
star
87

MPPITutorial

A codebase for running the MPPI algorithm on OpenAI gym style environments
Python
9
star
88

AutomotiveInteraction.jl

Driver modeling using the Interaction dataset
Julia
8
star
89

SBO_AIPPMS

Gaussian Process-based Adaptive Informative Path Planning with Multimodal Sensing
Julia
8
star
90

rltools

A collection of tools for conducting deep reinforcement learning experiments
Python
8
star
91

CUDD.jl

Wrapper for the CUDD library for decision diagrams
Julia
8
star
92

CEEM

Certainty-Equivalent Expectation Maximization: a scalable algorithm for system identification of partially observed systems
Python
8
star
93

VerifyGAN

Companion code for "Verification of Image-based Controllers Using Generative Models"
Julia
7
star
94

AutoUrban.jl

Additional packages for AutomotiveDrivingModels.jl in urban driving simulation
Julia
7
star
95

NeuralVerification-CARS-Workshop

CARS workshop on NeuralVerification.jl
Jupyter Notebook
7
star
96

OVERTVerify.jl

Jupyter Notebook
7
star
97

Gradescope.jl

Julia interface for Gradescope autograding
Julia
7
star
98

SatelliteTasking.jl

Satellite Satellite Tasking and Planning Tools and Analysis
Julia
7
star
99

CrossEntropyMethod.jl

An implementation of the cross entropy method that works well for time series
Julia
7
star
100

PerceptionAdaptiveStressTesting

Python
6
star