• Stars
    star
    370
  • Rank 112,684 (Top 3 %)
  • Language
    Python
  • License
    Other
  • Created about 1 year ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

[CoRL'23] Parting with Misconceptions about Learning-based Vehicle Motion Planning

tuPlangarage

A Framework for Vehicle Motion Planning Research

Paper | Supplementary | Talk | Slides | Video


Parting with Misconceptions about Learning-based Vehicle Motion Planning
Daniel Dauner1,2, Marcel Hallgarten1,3, Andreas Geiger1,2, and Kashyap Chitta1,2
1 University of Tübingen, 2 Tübingen AI Center, 3 Robert Bosch GmbH

This repo is intended to serve as a starting point for vehicle motion planning research on nuPlan. We provide a publicly accesible configuration for validation, comprehensive set of baselines, and pre-trained planning models.


teaser.mp4

News

  • 20 Aug, 2023: We renamed our repository to "tuPlan Garage" due to trademark conflicts.
  • 01 Aug, 2023: We released the code for GC-PGP!
  • 26 Jun, 2023: We released our supplementary material and the code for PDM-Closed.
  • 14 Jun, 2023: We released our paper on arXiv.
  • 2 Jun, 2023: Our approach won the 2023 nuPlan Challenge!

Overview

  • The release of nuPlan marks a new era in vehicle motion planning research, offering the first large-scale real-world dataset and evaluation schemes requiring both precise short-term planning and long-horizon ego-forecasting. Existing systems struggle to simultaneously meet both requirements.

  • Indeed, we find that these tasks are fundamentally misaligned and should be addressed independently.

  • We further assess the current state of closed-loop planning in the field, revealing the limitations of learning-based methods in complex real-world scenarios and the value of simple rule-based priors such as centerline selection through lane graph search algorithms.

  • More surprisingly, for the open-loop sub-task, we observe that the best results are achieved when using only this centerline as scene context (i.e., ignoring all information regarding the map and other agents).

  • Combining these insights, we propose an extremely simple and efficient planner which outperforms an extensive set of competitors, winning the nuPlan planning challenge 2023.


Contributing

If you consider contributing to tuPlan Garage, make sure to check out our Contribution Guidelines

Method

We decompose the process of determining a safe and comfortable trajectory into two sub-tasks: (1) planning the short-term motion, and (2) accurately forecasting the long-term ego trajectory. While the former primarily impacts closed-loop performance, the latter is essential for the open-loop task. Our method employs a rule-based predictive planner to generate a trajectory proposal, and a learned ego-forecasting module that refines the trajectory with a particular emphasis on long-term forecasting.

Results

Planning results on the proposed Val14 benchmark. Please refer to the paper for more details.

Method Representation CLS-R ↑ CLS-NR ↑ OLS ↑ Time (ms) ↓
Urban Driver* Polygon 44 45 76 64
GC-PGP Graph 56 55 83 100
PlanCNN Raster 72 73 64 43
IDM Centerline 77 76 38 27
PDM-Open Centerline 54 50 86 7
PDM-Closed Centerline 92 93 44 91
PDM-Hybrid Centerline 92 93 84 96
Log Replay GT 80 94 100 -

*Open-loop reimplementation of Urban Driver

To Do

  • Additional baselines
  • Visualization scripts
  • Contribution guide
  • ML planners code & checkpoints
  • Supplementary material, video, slides
  • Val14 benchmark
  • Installation tutorial
  • PDM-Closed release
  • Initial repo & main paper

Getting started

1. Installation

To install tuPlan Garage, please follow these steps:

  • setup the nuPlan dataset (described here) and install the nuPlan devkit (see here)
  • download tuPlan Garage and move inside the folder
git clone https://github.com/autonomousvision/tuplan_garage.git && cd tuplan_garage
  • make sure the environment you created when installing the nuplan-devkit is activated
conda activate nuplan
  • install the local tuplan_garage as a pip package
pip install -e .
  • add the following environment variable to your ~/.bashrc
NUPLAN_DEVKIT_ROOT="$HOME/nuplan-devkit/"

2. Training

When running a training, you have to add the hydra.searchpath for the tuplan_garage correctly. Note: since hydra does not yet support appending to lists (see here), you have to add the original searchpaths in the override. Training scripts can be run with the scripts found in /scripts/training/. Before training from an already existing cache, please check this issue. You can find our trained models here.

3. Evaluation

Same as for the training, when running an evaluation, you have to add the hydra.searchpath for the tuplan_garage correctly. The example below runs an evaluation of the pdm_closed_planner on the val14_split, both of which are part of the tuplan_garage

python $NUPLAN_DEVKIT_ROOT/nuplan/planning/script/run_simulation.py \
+simulation=closed_loop_nonreactive_agents \
planner=pdm_closed_planner \
scenario_filter=val14_split \
scenario_builder=nuplan \
hydra.searchpath="[pkg://tuplan_garage.planning.script.config.common, pkg://tuplan_garage.planning.script.config.simulation, pkg://nuplan.planning.script.config.common, pkg://nuplan.planning.script.experiments]"

You can find exemplary shells scripts in /scripts/simulation/

Contact

If you have any questions or suggestions, please feel free to open an issue or contact us ([email protected]).

Citation

If you find tuPlan Garage useful, please consider giving us a star 🌟 and citing our paper with the following BibTeX entry.

@article{Dauner2023ARXIV,
  title={Parting with Misconceptions about Learning-based Vehicle Motion Planning},
  author={Dauner, Daniel and Hallgarten, Marcel and Geiger, Andreas and Chitta, Kashyap},
  journal={arXiv},
  volume={2306.07962},
  year={2023}
}

Disclaimer

tuPlan Garage includes code from Motional's nuplan-devkit. We are not affiliated with Motional, and the repository is not published, maintained or otherwise related to Motional.

More Repositories

1

sdfstudio

A Unified Framework for Surface Reconstruction
Python
1,861
star
2

occupancy_networks

This repository contains the code for the paper "Occupancy Networks - Learning 3D Reconstruction in Function Space"
Python
1,454
star
3

giraffe

This repository contains the code for the CVPR 2021 paper "GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields"
Python
1,227
star
4

stylegan-t

[ICML'23] StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis
Python
1,122
star
5

transfuser

[PAMI'23] TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving; [CVPR'21] Multi-Modal Fusion Transformer for End-to-End Autonomous Driving
Python
957
star
6

stylegan-xl

[SIGGRAPH'22] StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets
Python
939
star
7

projected-gan

[NeurIPS'21] Projected GANs Converge Faster
Python
876
star
8

unimatch

[TPAMI'23] Unifying Flow, Stereo and Depth Estimation
Python
855
star
9

convolutional_occupancy_networks

[ECCV'20] Convolutional Occupancy Networks
Python
792
star
10

differentiable_volumetric_rendering

This repository contains the code for the CVPR 2020 paper "Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision"
Python
782
star
11

mip-splatting

[CVPR'24 Oral] Mip-Splatting: Alias-free 3D Gaussian Splatting
Python
700
star
12

monosdf

[NeurIPS'22] MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction
Python
535
star
13

shape_as_points

[NeurIPS'21] Shape As Points: A Differentiable Poisson Solver
Python
518
star
14

unisurf

[ICCV'21] UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction
Python
410
star
15

graf

Official code release for "GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis"
Jupyter Notebook
393
star
16

kitti360Scripts

This repository contains utility scripts for the KITTI-360 dataset.
Python
353
star
17

neat

[ICCV'21] NEAT: Neural Attention Fields for End-to-End Autonomous Driving
Python
291
star
18

gaussian-opacity-fields

Gaussian Opacity Fields for Efficient and Compact Surface Reconstruction in Unbounded Scenes
Python
285
star
19

occupancy_flow

This repository contains the code for the ICCV 2019 paper "Occupancy Flow - 4D Reconstruction by Learning Particle Dynamics"
Python
207
star
20

plant

[CoRL'22] PlanT: Explainable Planning Transformers via Object-Level Representations
Python
192
star
21

factor-fields

[SIGGRAPH 2023] We provide a unified formula for neural fields (Factor Fields) and a novel dictionary factorization (Dictionary Fields)
Jupyter Notebook
183
star
22

voxgraf

Official code release for VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids
Python
123
star
23

carla_garage

[ICCV'23] Hidden Biases of End-to-End Driving Models
Python
121
star
24

texture_fields

This repository contains code for the paper 'Texture Fields: Learning Texture Representations in Function Space'.
Python
113
star
25

sledge

SLEDGE: Synthesizing Simulation Environments for Driving Agents with Generative Models
105
star
26

kitti360LabelTool

JavaScript
103
star
27

counterfactual_generative_networks

[ICLR'21] Counterfactual Generative Networks
Python
102
star
28

gta

[ICLR'24] GTA: A Geometry-Aware Attention Mechanism for Multi-view Transformers
Python
95
star
29

murf

[CVPR'24] MuRF: Multi-Baseline Radiance Fields
Python
84
star
30

controllable_image_synthesis

Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis, CVPR 2020
Python
69
star
31

king

[ECCV'22] KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients
Python
61
star
32

handheld_svbrdf_geometry

On Joint Estimation of Pose, Geometry and svBRDF from a Handheld Scanner, CVPR2020
Python
57
star
33

navsim

NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation
Python
52
star
34

connecting_the_dots

This repository contains the code for the paper "Connecting the Dots: Learning Representations for Active Monocular Depth Estimation" https://avg.is.tuebingen.mpg.de/publications/riegler2019cvpr
Python
51
star
35

frequency_bias

Official code for "On the Frequency Bias of Generative Models", NeurIPS 2021
Python
39
star
36

data_aggregation

This repository contains the code for the CVPR 2020 paper "Exploring Data Aggregation in Policy Learning for Vision-based Urban Autonomous Driving"
Python
38
star
37

good

[ICLR'23] GOOD: Exploring Geometric Cues for Detecting Objects in an Open World
Python
36
star
38

campari

[3DV'21] CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields
Python
29
star
39

autonomousvision.github.io

Blog of the Autonomous Vision Group at MPI-IS Tübingen and University of Tübingen.
HTML
19
star
40

visual_abstractions

6
star
41

slides

Slide repository of the Autonomous Vision Group at MPI-IS Tübingen and University of Tübingen.
CSS
2
star
42

similarity_reconstruction

This code is based on the paper Exploiting Object Similarity in 3D Reconstruction.
C++
1
star
43

slow_flow

This code is based on the paper Slow Flow: Exploiting High-Speed Cameras for Accurate and Diverse Optical Flow Reference Data.
C++
1
star