• Stars
    star
    218
  • Rank 181,777 (Top 4 %)
  • Language
    Python
  • License
    MIT License
  • Created over 4 years ago
  • Updated over 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Siam R-CNN two-stage re-detector for visual object tracking

Siam R-CNN: Visual Tracking by Re-Detection

Paul Voigtlaender, Jonathon Luiten, Philip H.S. Torr, Bastian Leibe

The corresponding project page can be found here: https://www.vision.rwth-aachen.de/page/siamrcnn

This software is written in Python3 and powered by TensorFlow 1.

We borrow a lot of code from TensorPack's Faster R-CNN example: https://github.com/tensorpack/tensorpack/tree/master/examples/FasterRCNN

Installation

Download necessary libraries

Here we will put all external libraries and this repository into /home/${USERNAME}/vision and use pip to install common libraries

mkdir /home/${USERNAME}/vision
cd /home/${USERNAME}/vision

git clone https://github.com/VisualComputingInstitute/SiamR-CNN.git
git clone https://github.com/pvoigtlaender/got10k-toolkit.git
git clone https://github.com/tensorpack/tensorpack.git

cd tensorpack
git checkout d24a9230d50b1dea1712a4c2765a11876f1e193c
cd ..

pip3 install cython
pip3 install tensorflow-gpu==1.15
pip3 install wget shapely msgpack msgpack_numpy tabulate xmltodict pycocotools opencv-python tqdm zmq annoy

Add libraries to your PYTHONPATH

export PYTHONPATH=${PYTHONPATH}:/home/${USERNAME}/vision/got10k-toolkit/:/home/${USERNAME}/vision/tensorpack/

Make Folder for models and logs and download pre-trained model

cd SiamR-CNN/
mkdir train_log
cd train_log
wget --no-check-certificate -r -nH --cut-dirs=2 --no-parent --reject="index.html*" https://omnomnom.vision.rwth-aachen.de/data/siamrcnn/hard_mining3/
cd ..

Evaluation

For evaluation, first set the path to the dataset on which you want to evaluate in tracking/do_tracking.py, e.g.

OTB_2015_ROOT_DIR = '/data/otb2015/'

Then run tracking/do_tracking.py and specify the dataset you want to evaluate on using the main function for this dataset using e.g. --main main_otb

python3 tracking/do_tracking.py --main main_otb

The result will then be written to tracking_data/results/

Training

Download the pre-trained Mask R-CNN model from http://models.tensorpack.com/FasterRCNN/COCO-MaskRCNN-R101FPN9xGNCasAugScratch.npz

Now change the paths to the training datasets in config.py, e.g.

_C.DATA.IMAGENET_VID_ROOT = "/globalwork/data/ILSVRC_VID/ILSVRC/"

there you can also enable and disable different datasets, e.g.

_C.DATA.IMAGENET_VID = True

To run the main training (without hard example mining):

python3 train.py --load /path/to/COCO-R101FPN-MaskRCNN-ScratchGN.npz

Hints about the code

In the code, we sometimes use the terminology "ThreeStageTracker" or three stages. This refers to the Tracklet Dynamic Programming Algorithm (TDPA).

In order to make the code more readable, we removed some parts before publishing. If there's an important feature which you are missing, please write us an email at [email protected]

In the current version of the code, the functions to pre-compute the features for hard example mining are not available, but we can share the pre-computed data on request.

References

If you find this code useful, please cite

Siam R-CNN: Visual Tracking by Re-Detection
Paul Voigtlaender, Jonathon Luiten, Philip H.S. Torr, Bastian Leibe.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

More Repositories

1

triplet-reid

Code for reproducing the results of our "In Defense of the Triplet Loss for Person Re-Identification" paper.
Python
764
star
2

TrackR-CNN

TrackR-CNN baseline method for Multi-Object Tracking and Segmentation (MOTS)
Python
520
star
3

diffusion-e2e-ft

Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think
Python
275
star
4

mots_tools

Tools for evaluating and visualizing results for the Multi Object Tracking and Segmentation (MOTS) task
Python
222
star
5

2D_lidar_person_detection

Person detector for 2D range data. Code release for Self-Supervised Person Detection in 2D Range Data using a Calibrated Camera (https://arxiv.org/abs/2012.08890)
Python
161
star
6

vkitti3D-dataset

Python
100
star
7

3d-semantic-segmentation

This work is based on our paper Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds, which is appeared at the IEEE International Conference on Computer Vision (ICCV) 2017, 3DRMS Workshop.
Python
98
star
8

towards-reid-tracking

Code for the paper "Towards a Principled Integration of Multi-Camera Re-Identification and Tracking through Optimal Bayes Filters"
Python
82
star
9

DR-SPAAM-Detector

DR-SPAAM: A Spatial-Attention and Auto-regressive Model for Person Detection in 2D Range Data
Python
78
star
10

ShapePriors_GCPR16

C++
47
star
11

DROW

All code related to the "DROW: Real-Time Deep Learning based Wheelchair Detection in 2D Range Data" paper
Jupyter Notebook
41
star
12

Person_MinkUNet

Person-MinkUNet. Winner of JRDB 3D detection challenge in JRDB-ACT Workshop at CVPR 2021. https://arxiv.org/abs/2107.06780
Python
20
star
13

BiternionNets-ROS

An implementation of BiternionNets for ROS, ready to run on a robot.
Python
13
star
14

mots_trackingonly_tools

Tools for Challenge 3: Tracking Only (MOT+KITTI) of MOTChallenge 2020
Python
8
star
15

Beacon8

A Torch-inspired library for high-level deep learning with Theano.
Python
5
star
16

RovinaSemanticSegmentation

Semantic segmentation code for the ROVINA project.
C++
4
star
17

CROWDBOT_perception

This is the perception pipeline for the CROWDBOT project, featuring person detection and tracking from multi-sensor modalities.
C
3
star
18

cityscapes-util

Utility toolbox for dealing with the CityScapes dataset.
Python
3
star
19

omni3d-rgbd

2
star
20

PARIS-sem-seg

A straight forward network for semantic segmentation in TensorFlow
Python
2
star
21

ROS-laserdumper

Dump ROS LaserScan data into csv files.
C++
1
star