Recurrent Neural Networks for Semantic Instance Segmentation
See the paper in arXiv here.
Installation
- Clone the repo:
git clone https://github.com/imatge-upc/rsis.git- Install requirements
pip install -r requirements.txt - Install PyTorch 0.2 (choose the whl file according to your setup):
pip install http://download.pytorch.org/whl/cu80/torch-0.2.0.post3-cp27-cp27mu-manylinux1_x86_64.whl
pip install torchvision- Compile COCO Python API and add it to your
PYTHONPATH:
cd src/coco/PythonAPI;
make# Run from the root directory of this project
export PYTHONPATH=$PYTHONPATH:./src/coco/PythonAPIData
Pascal VOC 2012
- Download Pascal VOC 2012:
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
tar -xvf VOCtrainval_11-May-2012.tar- Download additional VOC annotations from Hariharan et al. Semantic contours from inverse detectors. ICCV 2011:
# berkeley augmented Pascal VOC
wget http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/semantic_contours/benchmark.tgz # 1.3 GB
tar zxvf benchmark.tgz- Create a merged dataset out of the two sets of images and annotations:
python src/dataloader/pascalplus_gen.py --voc_dir /path/to/pascal --contours_dir /path/to/additional/dataset --vocplus_dir /path/to/merged
- Precompute instance and semantic segmentation masks & ground truth files in COCO format:
python src/dataloader/pascal_precompute.py --split train --pascal_dir /path/to/merged
You must run this three times for the different splits (train, val and test).
Point args.pascal_dir to /path/to/merged.
CVPPP
Download the training CVPPP dataset from their website. In our case we just worked with the A1 dataset. Extract the A1 package and point args.leaves_dir to this folder. To obtain the test set for evaluation you will have to contact the organizers.
Cityscapes
Download the Cityscapes dataset from their website. Extract the images and the labels into the same directory and point args.cityscapes_dir to it.
Training
- Train the model with
python train.py -model_name model_name. Checkpoints and logs will be saved under../models/model_name. - Other arguments can be passed as well. For convenience, scripts to train with typical parameters are provided under
scripts/. - Visdom can be enabled to monitor training losses and outputs:
- First run the visdom server with
python -m visdom.server. - Run training with the
--visdomflag. Navigate tolocalhost:8097to visualize training curves.
- First run the visdom server with
- Plot loss curves at any time with
python plot_curves.py -model_name model_name.
Evaluation
We provide bash scripts to display results and evaluate models for the three datasets. You can find them under the scripts folder.
In the case of cityscapes, the evaluation bash script will generate the results in the appropiate format to use the official evaluation code.
For CVPPP, the evaluation bash script will generate the results in the appropiate format to use the evaluation scripts that are provided with the dataset.
Pretrained models
Download weights for models trained with:
Extract and place the obtained folder under models directory.
You can then run evaluation scripts with the downloaded model by setting args.model_name to the name of the folder.
Contact
For questions and suggestions use the issues section or send an e-mail to [email protected]
Additional notes to GPI users
Helpful commands to train on the GPI cluster and get visualizations in your browser:
- Start server: with
srun --tunnel $UID:$UID python -m visdom.server -port $UID. - Check the node where the server launched and (eg.
c3). - Run training with:
srun --gres=gpu:1,gmem:12G --mem=10G python train.py --visdom -port $UID -server http://c3
Notice that the port and the server must match the ones used in the previous run.
echo $UIDto know which port you are using.- ssh tunnel (run this in local machine):
ssh -L 8889:localhost:YOUR_UID -p2222 [email protected]. - Navigate to
localhost:8889in your browser locally.