• Stars
    star
    214
  • Rank 184,678 (Top 4 %)
  • Language
    Python
  • Created almost 8 years ago
  • Updated over 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Multi-view 3D Models from Single Images with a Convolutional Network

mv3d

alt tag Source code accompanying the ECCV'16 paper "Multi-view 3D Models from Single Images with a Convolutional Network" by M. Tatarchenko, A. Dosovitskiy and T. Brox http://lmb.informatik.uni-freiburg.de/people/tatarchm/mv3d/. The models implemented here are slightly different from those described in the paper, so we do not guarantee both quantitative and qualitative results to be exactly the same.

Dependencies

  • Tensorflow 0.12
  • Panda3D
  • NumPy
  • SciPy

Data

The networks were trained on a subset of the ShapeNet dataset containing 3D models of cars http://shapenet.cs.stanford.edu/. If you want to reproduce our results, you need to get the models. Follow the download instructions from the official website. Unzipped models should be placed in the 'data/obj_cars' folder in the following structure: data/obj_cars/model_id/{model.obj, model.mtl}. By default our rendering engine uses the .bam format, so after downloading the models you need to convert them. This can be done by subsequently applying obj2egg and egg2bam utilities, which come as a part of Panda3D package.

For the background experiment we used a subset of the ImageNet validation set. Those should be placed in data/bg_imagenet/val/ILSVRC2012_preprocessed_val_xxxxxxxx.JPEG, where xxxxxxxx is an 8-digit image id (in the range 00000001 - 00050000).

Usage

Run download_data.py first to download the pre-rendered version of the test set and the pre-trained network snapshots. We provide 3 pre-trained networks:

  • nobg_nodm.py - RGB input without background -> RGB output without background
  • nobg_dm.py - RGB input without background -> RGB-D output without background
  • bg_nodm.py - RGB input with background -> RGB output without background

You can run every script in train/test mode by uncommenting the corresponding parts of the script. When you run the script in the train mode, the realtime renderer is used to generate training data on the fly. To speed up the training process, you might want to implement a rendering app with multiple instances of the renderer running in parallel.

License and Citation

All code is provided for research purposes only and without any warranty. Any commercial use requires our consent. When using the code in your research work, please cite the following paper:

@InProceedings{TDB16a,
author       = "M. Tatarchenko and A. Dosovitskiy and T. Brox",
title        = "Multi-view 3D Models from Single Images with a Convolutional Network",
booktitle    = "European Conference on Computer Vision (ECCV)",
year         = "2016"
}

More Repositories

1

flownet2

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
C++
1,004
star
2

hand3d

Network estimating 3D Handpose from single color images
Python
801
star
3

demon

DeMoN: Depth and Motion Network
Python
574
star
4

freihand

A dataset for estimation of hand pose and shape from single color images.
Python
382
star
5

deeptam

DeepTAM: Deep Tracking and Mapping https://lmb.informatik.uni-freiburg.de/people/zhouh/deeptam/
Python
233
star
6

rgbd-pose3d

3D Human Pose Estimation in RGBD Images for Robotic Task Learning
Python
198
star
7

flownet2-docker

Dockerfile and runscripts for FlowNet 2.0 (estimation of optical flow)
Shell
158
star
8

netdef_models

Repository for different network models related to flow/disparity (ECCV 18)
Python
157
star
9

ogn

Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs
C++
155
star
10

orion

ORION: Orientation-boosted Voxel Nets for 3D Object Recognition
MATLAB
111
star
11

what3d

What Do Single-view 3D Reconstruction Networks Learn?
Python
98
star
12

dispnet-flownet-docker

Dockerfile and runscripts for DispNet and FlowNet1 (estimation of disparity and optical flow)
Shell
87
star
13

Unet-Segmentation

The U-Net Segmentation plugin for Fiji (ImageJ)
Java
87
star
14

robustmvd

Repository for the Robust Multi-View Depth Benchmark
Python
74
star
15

contra-hand

Code in conjunction with the publication 'Contrastive Representation Learning for Hand Shape Estimation'
Python
53
star
16

Multimodal-Future-Prediction

The official repository for the CVPR 2019 paper "Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction"
Python
47
star
17

lmbspecialops

A collection of tensorflow ops
C++
46
star
18

FLN-EPN-RPN

This repository contains the source code of the CVPR 2020 paper: "Multimodal Future Localization and Emergence Prediction for Objects in Egocentric View with a Reachability Prior"
Python
32
star
19

flow_rl

Python
28
star
20

netdef-docker

DispNet3, FlowNet3, FlowNetH, SceneFlowNet -- in Docker
Shell
28
star
21

caffe-unet-docker

The U-Net Segmentation server (caffe_unet) for Docker
Shell
27
star
22

Contrastive-Future-Trajectory-Prediction

The official repository of the ICCV paper "On Exposing the Challenging Long Tail in Future Prediction of Traffic Actors"
Python
25
star
23

locov

Localized Vision-Language Matching for Open-vocabulary Object Detection
Python
19
star
24

unsup-car-dataset

Unsupervised Generation of a Viewpoint Annotated Car Dataset from Videos
MATLAB
19
star
25

FreiPose-docker

FreiPose: A Deep Learning Framework for Precise Animal Motion Capture in 3D Spaces
Dockerfile
18
star
26

optical-flow-2d-data-generation

Caffe(v1)-compatible codebase to generate optical flow training data on-the-fly; used for the IJCV 2018 paper "What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?" (http://dx.doi.org/10.1007/s11263-018-1082-6)
C++
18
star
27

autodispnet

Code for AutoDispNet (ICCV 2019)
Python
17
star
28

cv-exercises

Python
15
star
29

spr-exercises

Jupyter Notebook
12
star
30

td-or-not-td

Code for the paper "TD or not TD: Analyzing the Role of Temporal Differencing in Deep Reinforcement Learning", Artemij Amiranashvili, Alexey Dosovitskiy, Vladlen Koltun and Thomas Brox, ICLR 2018
Python
12
star
31

sf2se3

Repository for SF2SE3: Clustering Scene Flow into SE(3)-Motions via Proposal and Selection
Python
10
star
32

ovqa

Python
10
star
33

understanding_flow_robustness

Official repository for "Towards Understanding Adversarial Robustness of Optical Flow Networks" (CVPR 2022)
Python
9
star
34

neural-point-cloud-diffusion

Official repository for "Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation"
Python
9
star
35

ldce

Official repository for "Latent Diffusion Counterfactual Explanations"
Python
9
star
36

PreFAct

Code and Models for the paper "Learning Representations for Predicting Future Activities"
8
star
37

ROS-packages

A collection of ROS packages for LMB software; DispNet(1+3), FlowNet2, etc.
C++
7
star
38

FreiPose

C++
7
star
39

diffusion-for-ood

Official repository for "Diffusion for Out-of-Distribution Detection on Road Scenes and Beyond". Coming soon.
Python
5
star
40

tfutils

tfutils is a set of tools for training networks with tensorflow
Python
5
star
41

FreiCalib

C++
5
star
42

netdef_slim

A python wrapper for tf to ease creation of network definitions.
Python
4
star
43

iRoCS-Toolbox

n-D Image Analysis libraries and tools
C++
4
star
44

rohl

Python
3
star
45

RecordTool

Python
2
star
46

tree-planting

Official repository for "Climate-sensitive Urban Planning Through Optimization of Tree Placements"
Python
2
star
47

ade-ood

Official repo for the ADE-OoD benchmark.
Python
1
star