• Stars
    star
    639
  • Rank 67,938 (Top 2 %)
  • Language
    Python
  • License
    Other
  • Created about 5 years ago
  • Updated almost 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

UPSNet: A Unified Panoptic Segmentation Network

UPSNet: A Unified Panoptic Segmentation Network

Introduction

UPSNet is initially described in a CVPR 2019 oral paper.

Disclaimer

This repository is tested under Python 3.6, PyTorch 0.4.1. And model training is done with 16 GPUs by using horovod. It should also work under Python 2.7 / PyTorch 1.0 and with 4 GPUs.

License

© Uber, 2018-2019. Licensed under the Uber Non-Commercial License.

Citing UPSNet

If you find UPSNet is useful in your research, please consider citing:

@inproceedings{xiong19upsnet,
    Author = {Yuwen Xiong, Renjie Liao, Hengshuang Zhao, Rui Hu, Min Bai, Ersin Yumer, Raquel Urtasun},
    Title = {UPSNet: A Unified Panoptic Segmentation Network},
    Conference = {CVPR},
    Year = {2019}
}

Main Results

COCO 2017 (trained on train-2017 set)

test split PQ SQ RQ PQTh PQSt
UPSNet-50 val 42.5 78.0 52.4 48.5 33.4
UPSNet-101-DCN test-dev 46.6 80.5 56.9 53.2 36.7

Cityscapes

PQ SQ RQ PQTh PQSt
UPSNet-50 59.3 79.7 73.0 54.6 62.7
UPSNet-101-COCO (ms test) 61.8 81.3 74.8 57.6 64.8

Requirements: Software

We recommend using Anaconda3 as it already includes many common packages.

Requirements: Hardware

We recommend using 4~16 GPUs with at least 11 GB memory to train our model.

Installation

Clone this repo to $UPSNet_ROOT

Run init.sh to build essential C++/CUDA modules and download pretrained model.

For Cityscapes:

Assuming you already downloaded Cityscapes dataset at $CITYSCAPES_ROOT and TrainIds label images are generated, please create a soft link by ln -s $CITYSCAPES_ROOT data/cityscapes under UPSNet_ROOT, and run init_cityscapes.sh to prepare Cityscapes dataset for UPSNet.

For COCO:

Assuming you already downloaded COCO dataset at $COCO_ROOT and have annotations and images folders under it, please create a soft link by ln -s $COCO_ROOT data/coco under UPSNet_ROOT, and run init_coco.sh to prepare COCO dataset for UPSNet.

Training:

python upsnet/upsnet_end2end_train.py --cfg upsnet/experiments/$EXP.yaml

Test:

python upsnet/upsnet_end2end_test.py --cfg upsnet/experiments/$EXP.yaml

We provide serveral config files (16/4 GPUs for Cityscapes/COCO dataset) under upsnet/experiments folder.

Model Weights

The model weights that can reproduce numbers in our paper are available now. Please follow these steps to use them:

Run download_weights.sh to get trained model weights for Cityscapes and COCO.

For Cityscapes:

python upsnet/upsnet_end2end_test.py --cfg upsnet/experiments/upsnet_resnet50_cityscapes_16gpu.yaml --weight_path ./model/upsnet_resnet_50_cityscapes_12000.pth
python upsnet/upsnet_end2end_test.py --cfg upsnet/experiments/upsnet_resnet101_cityscapes_w_coco_16gpu.yaml --weight_path ./model/upsnet_resnet_101_cityscapes_w_coco_3000.pth

For COCO:

python upsnet/upsnet_end2end_test.py --cfg upsnet/experiments/upsnet_resnet50_coco_16gpu.yaml --weight_path model/upsnet_resnet_50_coco_90000.pth
python upsnet/upsnet_end2end_test.py --cfg upsnet/experiments/upsnet_resnet101_dcn_coco_3x_16gpu.yaml --weight_path model/upsnet_resnet_101_dcn_coco_270000.pth

More Repositories

1

deep-neuroevolution

Deep Neuroevolution
Python
1,616
star
2

PPLM

Plug and Play Language Model implementation. Allows to steer topic and attributes of GPT-2 models.
Python
1,102
star
3

go-explore

Code for Go-Explore: a New Approach for Hard-Exploration Problems
Python
547
star
4

PyTorch-NEAT

Python
526
star
5

LaneGCN

[ECCV2020 Oral] Learning Lane Graph Representations for Motion Forecasting
Python
476
star
6

sbnet

Sparse Blocks Networks
Python
430
star
7

differentiable-plasticity

Implementations of the algorithms described in Differentiable plasticity: training plastic networks with gradient descent, a research paper from Uber AI Labs.
Python
394
star
8

DeepPruner

DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch (ICCV 2019)
Python
343
star
9

parallax

Tool for interactive embeddings visualization
Python
270
star
10

learning-to-reweight-examples

Code for paper "Learning to Reweight Examples for Robust Deep Learning"
Python
269
star
11

jpeg2dct

C++
251
star
12

poet

Paired Open-Ended Trailblazer (POET) and Enhanced POET
Python
235
star
13

intrinsic-dimension

Jupyter Notebook
220
star
14

CoordConv

Python
208
star
15

atari-model-zoo

A binary release of trained deep reinforcement learning models trained in the Atari machine learning benchmark, and a software release that enables easy visualization and analysis of models, and comparison across training algorithms.
Jupyter Notebook
201
star
16

ape-x

This repo replicates the results Horgan et al obtained in "Distributed Prioritized Experience Replay"
Python
188
star
17

EvoGrad

Python
178
star
18

TuRBO

Python
159
star
19

safemutations

safemutations
C++
143
star
20

permute-quantize-finetune

Using ideas from product quantization for state-of-the-art neural network compression.
Python
143
star
21

deconstructing-lottery-tickets

Python
142
star
22

CRISP

Python
131
star
23

metropolis-hastings-gans

Python
112
star
24

GTN

Python
75
star
25

backpropamine

Train self-modifying neural networks with neuromodulated plasticity
Python
73
star
26

loss-change-allocation

Python
61
star
27

MARVIN

Uber's Multi-Agent Routing Value Iteration Network
Python
52
star
28

GOCC

Go
51
star
29

Synthetic-Petri-Dish

Python
42
star
30

RxThreadEffectChecker

Static checker for Rx Threading Effects, based on the Checker Framework
Java
35
star
31

Map-Elites-Evolutionary

Map-Elites based on Evolution Strategies
Python
29
star
32

D3G

Estimating Q(s,s') with Deep Deterministic Dynamics Gradients
Python
29
star
33

java-dependency-validator

Dependency validator detects runtime compatibility issues at build time
Java
23
star
34

vargp

Variational Auto-Regressive Gaussian Processes for Continual Learning
Python
20
star
35

normative-uncertainty

Python
15
star
36

Evolvability-ES

Python
14
star
37

brezel

Starlark
8
star
38

dispatch-optim

Constrainted based optimization
Python
8
star
39

ga-world-models

Python
7
star
40

FSDM

Code tor the SIGDIAL 2019 paper Flexibly-Structured Model for Task-Oriented Dialogues. It implements a deep learning end-to-end differentiable dialogue system model
Python
7
star
41

rl-controller-verification

Quadcopter Verification
Python
5
star
42

go-context-propagate

Go
4
star
43

last-diff-analyzer

A multi-language tool for checking semantic equivalence for code
Go
2
star
44

tailr

TAILR
Python
1
star
45

xplane-bazel-docker

Bazel Xplane
C++
1
star