• Stars
    star
    1,124
  • Rank 41,395 (Top 0.9 %)
  • Language
    Python
  • License
    Other
  • Created over 6 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A PyTorch port of the Neural 3D Mesh Renderer

Neural 3D Mesh Renderer (CVPR 2018)

This repo contains a PyTorch implementation of the paper Neural 3D Mesh Renderer by Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. It is a port of the original Chainer implementation released by the authors. Currently the API is the same as in the original implementation with some smalls additions (e.g. render using a general 3x4 camera matrix, lens distortion coefficients etc.). However it is possible that it will change in the future.

The library is fully functional and it passes all the test cases supplied by the authors of the original library. Detailed documentation will be added in the near future.

Requirements

Python 2.7+ and PyTorch 0.4.0.

The code has been tested only with PyTorch 0.4.0, there are no guarantees that it is compatible with older versions. Currently the library has both Python 3 and Python 2 support.

Note: In some newer PyTorch versions you might see some compilation errors involving AT_ASSERT. In these cases you can use the version of the code that is in the branch at_assert_fix. These changes will be merged into master in the near future.

Installation

You can install the package by running

pip install neural_renderer_pytorch

Since running install.py requires PyTorch, make sure to install PyTorch before running the above command.

Running examples

python ./examples/example1.py
python ./examples/example2.py
python ./examples/example3.py
python ./examples/example4.py

Example 1: Drawing an object from multiple viewpoints

Example 2: Optimizing vertices

Transforming the silhouette of a teapot into a rectangle. The loss function is the difference between the rendered image and the reference image.

Reference image, optimization, and the result.

Example 3: Optimizing textures

Matching the color of a teapot with a reference image.

Reference image, result.

Example 4: Finding camera parameters

The derivative of images with respect to camera pose can be computed through this renderer. In this example the position of the camera is optimized by gradient descent.

From left to right: reference image, initial state, and optimization process.

Citation

@InProceedings{kato2018renderer
    title={Neural 3D Mesh Renderer},
    author={Kato, Hiroharu and Ushiku, Yoshitaka and Harada, Tatsuya},
    booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2018}
}

More Repositories

1

msckf_mono

Monocular MSCKF ROS Node
C++
486
star
2

spherical-cnn

Demo code for the paper "Learning SO(3) Equivariant Representations with Spherical CNNs"
Python
289
star
3

EV-FlowNet

Code for the paper "EV-FlowNet: Self-Supervised Optical Flow for Event-based Cameras"
Python
167
star
4

event_feature_tracking

This repo contains MATLAB implementations of the event-based feature tracking methods described in "Event-based Feature Tracking with Probabilistic Data Association" and "Event-based Visual Inertial Odometry".
MATLAB
82
star
5

monocap

Code for MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior.
MATLAB
60
star
6

emvn

Demo source code for the paper "Equivariant Multi-View Networks".
Python
56
star
7

polar-transformer-networks

Demo source code for the paper "Esteves, C., Allen-Blanchette, C., Zhou, X. and Daniilidis, K, "Polar Transformer Networks", ICLR 2018.
Python
54
star
8

mvsec

Multi Vehicle Stereo Event Camera Dataset
Python
52
star
9

ffmpeg_image_transport

image transport that uses libavcodec for compression
C++
42
star
10

m3ed

M3ED Dataset
Python
39
star
11

penncosyvio

The PennCOSYVIO data set
37
star
12

swscnn

Demo source code for the paper "Spin-Weighted Spherical CNNs".
23
star
13

cluster_tutorials

Generic slurm tutorials that explain intricacies of the cluster as implemented
Shell
19
star
14

all-graphs-lead-to-rome

Graph Convolutional Networks for multi-image matching
Python
17
star
15

ffmpeg_image_transport_tools

ros package with tools for ffmpeg compressed images
C++
7
star
16

drocap

7
star
17

EvAC3D

Python
7
star
18

daniilidis-group.github.io

home page of the Daniilidis group
HTML
5
star
19

flex_sync

ros package for syncing variable number of topics
C++
3
star
20

downsampling_image_transport

ROS image transport downsampling images in time
C++
2
star
21

grasp_multicam

repo for UPenn's GRASP MultiCam data set
Python
1
star