BEVStereo
BEVStereo is a new multi-view 3D object detector using temporal stereo to enhance depth estimation.
Updates!!
- 【2022/09/22】 We released our paper on Arxiv.
- 【2022/08/24】 We submitted our result on nuScenes Detection Task and achieved the SOTA.
Quick Start
Installation
Step 0. Install pytorch(v1.9.0).
Step 1. Install MMDetection3D(v1.0.0rc4).
Step 2. Install requirements.
pip install -r requirements.txtStep 3. Install BEVDepth(gpu required).
python setup.py developData preparation
We use the same data format as BEVDepth, if you have processed it before, please skip.
Step 0. Download nuScenes official dataset.
Step 1. Symlink the dataset root to ./data/.
ln -s [nuscenes root] ./data/
The directory will be as follows.
BEVDepth
├── data
│ ├── nuScenes
│ │ ├── maps
│ │ ├── samples
│ │ ├── sweeps
│ │ ├── v1.0-test
| | ├── v1.0-trainval
Step 2. Prepare infos.
python scripts/gen_info.py
Step 3. Prepare depth gt.
python scripts/gen_depth_gt.py
Tutorials
Train.
python [EXP_PATH] --amp_backend native -b 8 --gpus 8
Eval.
python [EXP_PATH] --ckpt_path [CKPT_PATH] -e -b 8 --gpus 8
Benchmark
| Exp | Frames | EMA | CBGS | mAP | mATE | mASE | mAOE | mAVE | mAAE | NDS | weights |
|---|---|---|---|---|---|---|---|---|---|---|---|
| R50 | key + sweep4 | 0.3427 | 0.6560 | 0.2784 | 0.5982 | 0.5347 | 0.2228 | 0.4423 | github | ||
| R50 | key + sweep4 | √ | 0.3435 | 0.6585 | 0.2757 | 0.5792 | 0.5034 | 0.2163 | 0.4485 | github | |
| R50 | key + key | 0.3456 | 0.6589 | 0.2774 | 0.5500 | 0.4980 | 0.2278 | 0.4516 | github | ||
| R50 | key + key | √ | 0.3494 | 0.6671 | 0.2785 | 0.5606 | 0.4686 | 0.2295 | 0.4543 | github | |
| R50 | key + key | √ | 0.3576 | 0.6071 | 0.2684 | 0.4157 | 0.3928 | 0.2021 | 0.4902 | github | |
| R50 | key + key | √ | √ | 0.3721 | 0.5980 | 0.2701 | 0.4381 | 0.3672 | 0.1898 | 0.4997 | github |
Acknowledgments
This project exists thanks to all the people who instruct. @Haotian-Zh @xavierwu95 @Tai-Wang