Multi-Object-Tracking-Paper-List

Some new works could be found in the MOT16、MOT17 ranking list, but it seems that no work opening their source code in recent months, so I did not update the source code list. Sometimes, I think that the MOT benchmark may lead the researchers to a wrong direction. The MOT17 benchmark provides three kinds of detection results: DPM, Faster RCNN and SDP, if you analyse the results based on different detections, you would find that most trackers has similar performance on Faster RCNN and SDP detections. The trackers ranking well mostly rely on the good performance on DPM detections and they have similar performance with DeepSort when adopting Faster RCNN and SDP detections. So if you want to improve your MOTA score, focus on DPM result.

Multi-object tracking is a deeply explored but not successfully solved computer vision task. This filed needs more open sources and more standard evaluation metrics. Opening source code is not the characteristic for this field（maybe the large number of parameters waited to be tuned)...... This is a paper list for multi-object-tracking.

Datasets

Surveillance Scenarios

PETS2009 : An old dataset.
MOT dataset : A dataset for multi-person detection and tracking, mostly used.
UA-DETRAC : A dataset for multi-car detection and tracking.
AVSS2018 Challenge : AVSS2018 Challenge based on UA-DETRAC is opened!
DukeMTMC : A dataset for multi-camera multi-person tracking.
PoseTrack: A dataset for multi-person pose tracking.
NVIDIA AI CITY Challenge: Challenges including "Traffic Flow Analysis", "Anomaly Detection" and "Multi-sensor Vehicle Detection and Reidentification", you may find some insteresting codes on their Github repos
Vis Drone: Tracking videos captured by drone-mounted cameras.
JTA Dataset: A huge dataset for pedestrian pose estimation and tracking in urban scenarios created by exploiting the highly photorealistic video game Grand Theft Auto V developed by Rockstar North.
Path Track A new dataset with many scenes.
MOTS MOTS: Multi-Object Tracking and Segmentation. In CVPR 2019

Driving Scenarios

KITTI-Tracking : Multi-person or multi-car tracking dataset.
MOTS Multi-Object Tracking and Segmentation. In CVPR 2019
Apollo-Tracking 3D Lidar multi-object tracking.
Baidu Trajectory Interesting dataset for trajectory prediction for Autonomous drive, wait to be opened.

Overview

P Emami,PM Pardalos,L Elefteriadou,S Ranka "Machine Learning Methods for Solving Assignment Problems in Multi-Target Tracking" [paper]
Wenhan Luo, Junliang Xing, Anton Milan, Xiaoqin Zhang, Wei Liu, Xiaowei Zhao and Tae-Kyun Kim, "Multiple Object Tracking: A Literature Review" [paper]
A 101 slide [paper]

Evaluation Metric

CLEAR MOT : Bernardin, K. & Stiefelhagen, R. "Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metric" [paper]
IDF1 : Ristani, E., Solera, F., Zou, R., Cucchiara, R. & Tomasi, C. "Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking" [paper]
Evaluation Code: [Python]

Researcher

Computer Vision Group at RWTH Aachen University [webpage and source code]
Anton Milan [webpage and his source code]
Laura Leal-Taixé [webpage and her source code]
Dynamic Vision and Learning Group [webpage and their source code]
Longyin Wen [webpage and his source code]
UCF [webpage]
Some source codes in above webpage are not listed in below Open Source.Such as:
"segTrack"
"Exploiting Hierarchical Dense Structures on Hypergraphs for Multi-Object Tracking"
"Learning an image-based motion context for multiple people tracking"

Open Source

Online

Tracking-Objects-Points Zhou, Xingyi and Koltun, Vladlen and Kr{"a}henb{"u}hl, Philipp. Tracking Objects as Points[paper][code]
Towards-Realtime-MOT Zhongdao Wang, Liang Zheng, Yixuan Liu, Shengjin Wang. Towards Real-Time Multi-Object Tracking. [paper] [code]
Track-no-bnw Bergmann P, Meinhardt T, Lealtaixe L, et al. Tracking without bells and whistles[J]. (ICCV2020). [paper][code]
DeepMot Yihong Xu Yutong Ban Xavier Alameda-Pineda Radu Horaud, "DeepMOT:A Differentiable Framework for Training Multiple Object Trackers" [paper][code]
TrackR-CNN,MOTS Paul Voigtlaender and Michael Krause, "MOTS: Multi-Object Tracking and Segmentation" In CVPR2019 [paper][code]
Tracktor Philipp Bergmann, Tim Meinhardt, Laura Leal-Taixe "Tracking without bells and whistles" In Arxiv [paper][code]
DMAN Zhu, Ji and Yang, Hua and Liu, Nian and Kim, Minyoung and Zhang, Wenjun and Yang, Ming-Hsuan "Online Multi-Object Tracking with Dual Matching Attention Networks" [paper] [code] In ECCV2018
LSST Weitao Feng, Zhihao Hu, Wei Wu, Junjie Yan, Wanli Ouyang "Multi-Object Tracking with Multiple Cues and Switcher-Aware Classification" [paper] SOA tracker on MOT ranking list. No code now.
SST Sun. S., Akhtar, N., Song, H., Mian A., & Shah M. (2018). Deep Affinity Network for Multiple Object Tracking[paper][code]: Interesting work and expect the author to update their DPM tracking results on MOT17 benchmark.
MOTDT Long Chen, Haizhou Ai "Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-identification" in ICME 2018 [code][paper]!
TMPORT : E. Ristani and C. Tomasi. Tracking Multiple People Online and in Real Time. in ACCV 2014 [paper] [code]
MOT-RNN : Anton Milan, Seyed Hamid Rezatofighi, Anthony Dick, Konrad Schindler, Ian Reid "Online Multi-target Tracking using Recurrent Neural Networks"[paper] [code] In AAAI 2017.
DeepSort : Wojke, Nicolai and Bewley, Alex and Paulus, Dietrich "Simple Online and Realtime Tracking with a Deep Association Metric" [paper] [code] In ICIP 2017
Sort : Bewley, Alex and Ge, Zongyuan and Ott, Lionel and Ramos, Fabio and Upcroft, Ben "Simple Online and Realtime Tracking"[paper] [code] In ICIP 2016.
MDP : Yu Xiang, Alexandre Alahi, and Silvio Savarese "Learning to Track: Online Multi-Object Tracking by Decision Making " [paper] [code] In International Conference on Computer Vision (ICCV), 2015
CMOT : S. H. Bae and K. Yoon. "Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning" [paper] [code] In CVPR 2014
RCMSS : Mohamed A. Naiel1, M. Omair Ahmad, M.N.S. Swamy, Jongwoo Lim, and Ming-Hsuan Yang "Online Multi-Object Tracking Via Robust Collaborative Model and Sample Selection"[paper] [code] Computer Vision and Image Understanding 2016
MHT-DAM : Chanho Kim, Fuxin Li, Arridhana Ciptadi, James M. Rehg "Multiple Hypothesis Tracking Revisited"[paper] [code] In ICCV 2015
OMPTTH : Jianming Zhang, Liliana Lo Presti and Stan Sclaroff, "Online Multi-Person Tracking by Tracker Hierarchy," [paper] [code] Proc. Int. Conf. on Advanced Video and Signal Based Surveillance (AVSS), 2012.
SMOT : C. Dicle, O. Camps, M. Sznaier. "The Way They Move: Tracking Targets with Similar Appearance" [paper] [code] In ICCV, 2013.

Batch

muSSP Wang C, Wang Y, Wang Y, et al. muSSP: Efficient Min-cost Flow Algorithm for Multi-object Tracking[C]. neural information processing systems, 2019: 423-432. [paper][code]
TNT Gaoang Wang, Yizhou Wang, Haotian Zhang, Renshu Gu, Jenq-Neng Hwang "Exploit the Connectivity: Multi-Object Tracking with TrackletNet" [paper] [code]
NT Longyin Wen*, Dawei Du*, Shengkun Li, Xiao Bian, Siwei Lyu Learning Non-Uniform Hypergraph for Multi-Object Tracking, In AAAI 2019 [paper][code] Waited!
headTracking: Shun Zhang, Jinjun Wang, Zelun Wang, Yihong Gong,Yuehu Liu: "Multi-Target Tracking by Learning Local-to-Global Trajectory Models" in PR 2015 [paper] [code] seems like a repo.
IOU : E. Bochinski, V. Eiselein, T. Sikora. "High-Speed Tracking-by-Detection Without Using Image Information" [paper] [code] In International Workshop on Traffic and Street Surveillance for Safety and Security at IEEE AVSS 2017, 2017.
NMGC-MOT Andrii Maksai, Xinchao Wang, Franc¸ois Fleuret, and Pascal Fua "Non-Markovian Globally Consistent Multi-Object Tracking " [paper][code] In ICCV 2017
D2T Christoph Feichtenhofer, Axel Pinz, Andrew Zisserman, "Detect to Track and Track to Detect" [paper] [code] In ICCV 2017
H2T : Longyin Wen, Wenbo Li, Junjie Yan, Zhen Lei, Dong Yi, Stan Z. Li. "Multiple Target Tracking Based on Undirected Hierarchical Relation Hypergraph," [paper] [code] IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
LDCT : F. Solera, S. Calderara, R. Cucchiara "Learning to Divide and Conquer for Online Multi-Target Tracking" [paper] [code page 1] [code page 2] In Proceedings of International Converence on Computer Vision (ICCV), Santiago Cile, Dec 12-18, 2015
CEM : Anton Milan, Stefan Roth, Konrad Schindler "Continuous Energy Minimization for Multi-Target Tracking" [paper] [code] in pami 2014
OPCNF : Chari, Visesh and Lacoste-Julien, Simon and Laptev, Ivan and Sivic, Josef "On Pairwise Costs for Network Flow Multi-Object Tracking" [paper] [code] In CVPR 2015
KSP : J. Berclaz, F. Fleuret, E. Türetken and P. Fua "Multiple Object Tracking using K-Shortest Paths Optimization" [paper] [code] IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011.
GMCP : Amir Roshan Zamir, Afshin Dehghan, and Mubarak Shah "GMCP-Tracker: Global Multi-object Tracking Using Generalized Minimum Clique Graphs" [paper] [code] European Conference on Computer Vision (ECCV), 2012.

Driving Scenarios

MOTBeyondPixels Sarthak Sharma*, Junaid Ahmed Ansari*, J. Krishna Murthy, and K. Madhava Krishna Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Tracking In ICRA 2018 [paper][code]
CIWT Aljoˇsa Oˇsep, Alexander Hermans Combined Image and World-Space Tracking in Traffic Scenes In ICRA 2017 [paper] [code]

MCMT

DeepCC Ristani and C. Tomasi "Features for Multi-Target Multi-Camera Tracking and Re-Identification" In CVPR 2018 [paper] [code]
towards-reid-tracking Lucas Beyer∗ Stefan Breuers∗ "Towards a Principled Integration of Multi-Camera Re-Identification andTracking through Optimal Bayes Filters"[paper][code]

RGBD Tracking

DetTA Stefan Breuers, Lucas Beyer "Detection-Tracking for Efficient Person Analysis: The DetTA Pipeline" [paper][code]

Private Detection

POI : F. Yu, W. Li, Q. Li, Y. Liu, X. Shi, J. Yan. "POI: Multiple Object Tracking with High Performance Detection and Appearance Feature" [paper] [detection] In BMTT, SenseTime Group Limited, 2016

New papers

CVPR

2017

Eldar Insafutdinov, Mykhaylo Andriluka, Leonid Pishchulin, Siyu Tang, Evgeny Levinkov, Bjoern Andres, Bernt Schiele "Art Track: Articulated Multi-Person Tracking in the Wild" [paper]
Manmohan Chandraker, Paul Vernaza, Wongun Choi, Samuel Schulter "Deep Network Flow for Multi-Object Tracking" [paper]
Jeany Son, Mooyeol Baek, Minsu Cho, and Bohyung Han, "Multi-Object Tracking with Quadruplet Convolutional Neural Networks" [paper]

2018

Girdhar R, Gkioxari G, Torresani L, et al. Detect-and-Track: Efficient Pose Estimation in Videos[C].(CVPR2018)[paper][code]
Rolling Shutter and Radial Distortion Are Features for High Frame Rate Multi-Camera Tracking[paper]
Features for Multi-Target Multi-Camera Tracking and Re-Identification [paper]

2019

Self-Supervised Adaptation of High-Fidelity Face Models for Monocular Performance Tracking [paper]
Eliminating Exposure Bias and Metric Mismatch in Multiple Object Tracking [paper]

ICCV

A. Sadeghian, A. Alahi, S. Savarese, Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies [paper]
Andrii Maksai, Xinchao Wang, Franc¸ois Fleuret, and Pascal Fua "Non-Markovian Globally Consistent Multi-Object Tracking " [paper][code]
Christoph Feichtenhofer, Axel Pinz, Andrew Zisserman, "Detect to Track and Track to Detect" [paper] [code]
Qi Chu, Wanli Ouyang, Xiaogang Wang, Bin Liu, Nenghai Yu "Online Multi-Object Tracking Using CNN-Based Single Object Tracker With Spatial-Temporal Attention Mechanism" [paper]
Track-no-bnw Bergmann P, Meinhardt T, Lealtaixe L, et al. Tracking without bells and whistles[J]. (ICCV2020). [paper][code]

ECCV2018

Ren, Liangliang and Lu, Jiwen and Wang, Zifeng and Tian, Qi and Zhou, Jie "Collaborative Deep Reinforcement Learning for Multi-Object Tracking" [paper]
Kim, Chanho and Li, Fuxin and Rehg, James M "Multi-object Tracking with Neural Gating Using Bilinear LSTM" [paper]

New paper

M Fabbri, F Lanzi, S Calderara, A Palazzi "Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World" [paper] [[code]] Waited!
Cong Ma, Changshui Yang, Fan Yang, Yueqing Zhuang, Ziwei Zhang, Huizhu Jia, Xiaodong Xie "Trajectory Factory: Tracklet Cleaving and Re-connection by Deep Siamese Bi-GRU for Multiple Object Tracking" In ICME 2018 [paper]
Kuan Fang, Yu Xiang, Xiaocheng Li and Silvio Savarese "Recurrent Autoregressive Networks for Online Multi-Object Tracking" In IEEE Winter Conference on Applications of Computer Vision (WACV), 2018. [webpage]
Tharindu Fernando, Simon Denman, Sridha Sridharan, Clinton Fookes "Tracking by Prediction: A Deep Generative Model for Mutli-Person localisation and Tracking" In WACV 2018 [paper]

Multi-person Face Tracking

Shun Zhang, Yihong Gong, Jia-Bin Huang, Jongwoo Lim, Jinjun Wang, Narendra Ahuja and Ming-Hsuan Yang "Tracking Persons-of-Interest via Adaptive Discriminative Features" In ECCV 2016 [paper] [code]
Chung-Ching Lin, Ying Hung"A Prior-Less Method for Multi-Face Tracking in Unconstrained Videos" In CVPR 2018 [paper]

Multi-person Pose Tracking

Yuliang Xiu, Jiefeng Li, Haoyu Wang, Yinghong Fang, Cewu Lu "Pose Flow: Efficient Online Pose Tracking" [paper] Idea is interesting but the true source code is not opened.
Bin Xiao, Haiping Wu, and Yichen Wei "Simple Baselines for Human Pose Estimation and Tracking" [paper][code]
Girdhar R, Gkioxari G, Torresani L, et al. Detect-and-Track: Efficient Pose Estimation in Videos[C].(CVPR2018)[paper][code] Multi-Person Articulated Tracking With Spatial and Temporal Embeddings [paper]

Multi-camera Tracking

Rolling Shutter and Radial Distortion Are Features for High Frame Rate Multi-Camera Tracking[paper] CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification[paper]