3D Multi-Object Tracker

This project is developed for tracking multiple objects in 3D scene. The visualization code is from here. This is a re-produced and simplified version of paper. We have made some changes to support more detectors and adapt the HOTA metric. We also provided a simple guide to submit results to the KITTI test set.

Features

• Fast: currently, the codes can achieve 700 FPS using only CPU (not include detection and data op), can perform tracking on all kitti val sequence in several seconds.
• Support online, near online and global implementation. The overall framework of design is shown below:

Kitti Results

Car/Pedestrian tracking results on the Kitti tracking val seq [1,6,8,10,12,13,14,15,16,18,19] using second-iou, point-rcnn and pv-rcnn detections. We also followed the HOTA metric, and tuned the parameters by first considering the HOTA performance.

Online: CA-based KF + greedy matching. Near online: online + rescoring tracks in a temporal window. Global: online + rescoring tracks globally.

• 2022/9/23 Update CasTrack. The detections are obtained by CasA detector trained on KITTI 3D detection trainval set. Currently rank first on the KITTI tracking dataset 🔥!

• Update VirConvTrack (2023/3/7), the detections are obtained by VirConv-T detector trained on KITTI 3D detection trainval set.

Waymo Results

Vehicle/Pedestrian/Cyclist online tracking results, evaluated by MOTA/L2 metrics of Waymo benchmark.

• 2022/9/23 Update CasTrack-waymo, currently ranks third on Waymo tracking benchmark 🔥! You can download the CasA detections from here (Please register with waymo before downloading, as the Waymo open dataset is under strict non-commercial license).

Prepare data

You can download the Kitti tracking pose data from here, and you can download the point-rcnn, second-iou and pv-rcnn detections from here. You can download the CasA detections(including training & testing set) from here.

To run this code, you should organize Kitti tracking dataset as below:

# Kitti Tracking Dataset       
└── kitti_tracking
       ├── testing 
       |      ├──calib
       |      |    ├──0000.txt
       |      |    ├──....txt
       |      |    └──0028.txt
       |      ├──image_02
       |      |    ├──0000
       |      |    ├──....
       |      |    └──0028
       |      ├──pose
       |      |    ├──0000
       |      |    |    └──pose.txt
       |      |    ├──....
       |      |    └──0028
       |      |         └──pose.txt
       |      ├──label_02
       |      |    ├──0000.txt
       |      |    ├──....txt
       |      |    └──0028.txt
       |      └──velodyne
       |           ├──0000
       |           ├──....
       |           └──0028      
       └── training # the structure is same as testing set
              ├──calib
              ├──image_02
              ├──pose
              ├──label_02
              └──velodyne 

Detections

└── point-rcnn
       ├── training
       |      ├──0000
       |      |    ├──000001.txt
       |      |    ├──....txt
       |      |    └──000153.txt
       |      ├──...
       |      └──0020
       └──testing 

Requirements

python3
numpy
opencv
yaml

Quick start

• Please modify the dataset path and detections path in the yaml file to your own path.
• Then run python3 kitti_3DMOT.py config/online/pvrcnn_mot.yaml
• The results are automatically saved to evaluation/results/sha_key/data, and evaluated by HOTA metrics.

Notes

The evaluation codes are copied from Kitti.