akanimax/thr3ed_atom akanimax/thr3ed_atom

  • Stars
    star
    111
  • Rank 312,578 (Top 7 %)
  • Language
    Python
  • License
    MIT License
  • Created over 2 years ago
  • Updated 8 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
akanimax/thr3ed_atom
github

akanimax

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

ReLU Fields The Little Non-linearity That Could

โญ 3inFusion

[Work in progress] Showcase project for the thr3ed_atom package. We use a diffusion network to model semantic variations of a given single 3D scene which is self similar and stochastic. Following is a teaser of this concept: teaser The diffusion model is trained on fixed sized 3D crops of the original 3D voxel-grid (containing the object or the scene). At test-time, samples of original resolution are drawn using the trained model. This idea essentially replaces the GAN in 3inGAN with DIFFUSION. The code can be located under the thre3d_elements/thre3infusion directory.

Some results in Action:

G.T. 3D scene:

rendered_video.mp4

Random samples:

samples_75000.mp4

relu_fields

Official implementation of the paper: ReLU Fields: The Little Non-linearity That Could. The implementation uses the PyTorch framework.

GitHub Generic badge

teaser

Mitsuba 3 tutorial

Check out this Mitsuba 3 inverse rendering tutorial based on ReLU Fields.

Radiance Fields Experiments

hotdog

Data:

The data used for these experiments is available at this drive link. Download the synthetic_radiance_fields_dataset.zip and extract it in some folder on your disk. For easy access create a /data folder in this repo-root and extract the zip in something like /data/radiance_fields.

Run the optimization:

Running the optimization is super simple ๐Ÿ˜„. Just use the script train_sh_based_voxel_grid_with_posed_images.py python script with appropriate cmd-args. Following example runs the training on the Hotdog scene:

(relu_fields_env) user@machine:<repo_path>$ python train_sh_based_voxel_grid_with_posed_images.py -d data/radiance_fields/hotdog -o logs/rf/hotdog/

Render trained model:

Use the render_sh_based_voxel_grid.py script for creating a rotating/spiral 3D render of the trained models. Following are all the options that you can tweak in this render_script

Usage: render_sh_based_voxel_grid.py [OPTIONS]

Options:
  -i, --model_path FILE           path to the trained (reconstructed) model
                                  [required]
  -o, --output_path DIRECTORY     path for saving rendered output  [required]
  --overridden_num_samples_per_ray INTEGER RANGE
                                  overridden (increased) num_samples_per_ray
                                  for beautiful renders :)  [x>=1]
  --render_scale_factor FLOAT     overridden (increased) resolution (again :D)
                                  for beautiful renders :)
  --camera_path [thre360|spiral]  which camera path to use for rendering the
                                  animation
  --camera_pitch FLOAT            pitch-angle value for the camera for 360
                                  path animation
  --num_frames INTEGER RANGE      number of frames in the video  [x>=1]
  --vertical_camera_height FLOAT  height at which the camera spiralling will
                                  happen
  --num_spiral_rounds INTEGER RANGE
                                  number of rounds made while transitioning
                                  between spiral radii  [x>=1]
  --fps INTEGER RANGE             frames per second of the video  [x>=1]
  --help                          Show this message and exit.

Coming soon ...

The Geometry (occupancy), Pixel Fields (Images) and the Real-scene experiments will be setup in the thre3d_atom package soon.

BibTex:

@article{Karnewar2022ReLUFields,
    author    = {Karnewar, Animesh and Ritschel, Tobias and Wang, Oliver and Mitra, Niloy J.},
    title     = {ReLU Fields: The Little Non-linearity That Could},
    journal   = {Transactions on Graphics (Proceedings of SIGGRAPH),
    volume    = {41},
    number    = {4},
    year      = {2022},
    month     = july,
    pages     = {13:1--13:8},
    doi       = {10.1145/3528233.3530707},
}

Thanks

As always,
please feel free to open PRs/issues/suggestions here ๐Ÿ˜„.

cheers ๐Ÿป!
@akanimax ๐Ÿ˜Ž ๐Ÿค–

More Repositories

1

BMSG-GAN

[MSG-GAN] Any body can GAN! Highly stable and robust architecture. Requires little to no hyperparameter tuning. Pytorch Implementation
Python
630
star
2

T2F

T2F: text to face generation using Deep Learning
Python
546
star
3

pro_gan_pytorch

Unofficial PyTorch implementation of the paper titled "Progressive growing of GANs for improved Quality, Stability, and Variation"
Python
536
star
4

msg-stylegan-tf

MSG StyleGAN in tensorflow
Python
264
star
5

natural-language-summary-generation-from-structured-data

Implementation of the paper -> https://arxiv.org/abs/1709.00155. For converting information present in the form of structured data into natural language text
Python
183
star
6

Variational_Discriminator_Bottleneck

Implementation (with some experimentation) of the paper titled "VARIATIONAL DISCRIMINATOR BOTTLENECK: IMPROVING IMITATION LEARNING, INVERSE RL, AND GANS BY CONSTRAINING INFORMATION FLOW" (arxiv -> https://arxiv.org/pdf/1810.00821.pdf)
Python
152
star
7

msg-gan-v1

MSG-GAN: Multi-Scale Gradients GAN (Architecture inspired from ProGAN but doesn't use layer-wise growing)
Python
151
star
8

fagan

A variant of the Self Attention GAN named: FAGAN (Full Attention GAN)
Python
112
star
9

big-discriminator-batch-spoofing-gan

BMSG-GAN with more features
Python
45
star
10

pro_gan_pytorch-examples

Examples trained using the python pytorch package pro-gan-pth
Python
38
star
11

attn_gan_pytorch

python package for self-attention gan implemented as extension of PyTorch nn.Module. paper -> https://arxiv.org/abs/1805.08318
Python
19
star
12

my-deity

I worship the one true neural network architecture that can autonomously learn everything.
Jupyter Notebook
12
star
13

NLP2SQL

A research and review of techniques to provide a natural language interface to RDMS.
Jupyter Notebook
11
star
14

open-styleganv2-pytorch

Open source + Free for Commercial Use implementation of StyleGANv2 in pytorch
Python
7
star
15

capsule-network-TensorFlow

The impending concept of capsule networks has finally arrived at arXiv. link to the publication -> https://arxiv.org/abs/1710.09829 . In this repository, I'll create an implementation using TensorFlow from scratch as an exercise.
Jupyter Notebook
6
star
16

3inGAN

Python
5
star
17

GAN-understanding

Implements gans on toy datasets and preliminary ML datasets for showing certain aspects of convergence and stability. Tries to cover various loss functions defined over the years.
Jupyter Notebook
5
star
18

autoencoder-cifar-10

Implementing an auto-encoder for the cifar10 dataset
Jupyter Notebook
4
star
19

Homecoming

repository for mini-projects
Python
3
star
20

python_ai_project_template

A lightweight template for building AI-based prototype/research POCs in Python. My poison (DL framework :laugh: ) of choice is PyTorch!
Python
3
star
21

some-randon-gan-1

MSG-GAN with self attention. For MSG-GAN head to -> https://github.com/akanimax/MSG-GAN
Python
2
star
22

AI-Literature

A repository to store key research works from the past. It is also an attempt to structure and organize these research papers.
2
star
23

indian-celeb-gans

Various GANs trained on a dataset containing images of Indian Celebrities (procured by me).
Python
2
star
24

some-random-gan-2

More experimentation with the base MSG-GAN architecture. This includes the coord-conv layers in the architecture. For more info about MSG-GAN, head to -> https://github.com/akanimax/msg-stylegan-tf
Python
2
star
25

multithreaded-histogram-equalization-cpp

Explanatory Code for performing Histogram Equalization on Images for contrast improvement. The code uses OpenCV in C++ for image read/write and uses pthread for multithreading
C++
2
star
26

deep-reinforcement-learning

Project for studying and implementing the traditional RL algorithms and also the DL variants of the same.
Jupyter Notebook
1
star
27

dcgan_pytorch

GAN example created using the attn_gan_pytorch package -> https://github.com/akanimax/attn_gan_pytorch
Python
1
star
28

CL-3_lab_2017

repository for assignments of Computer Laboratory 3 - 2016
TeX
1
star
29

SVC2004-deep-learning

A deep learning based solution for the SVC2004 problem.
Jupyter Notebook
1
star
30

toxic-comment-identification-tensorflow

Data -> https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/data
Python
1
star
31

REST_MOVIE_TICKET_SYSTEM

A restful system implementing token based authentication for allowing users to book movie tickets online. Use of Play framework for scala
Scala
1
star
32

algorithms

A repository for collecting the coding implementations of some of the most famous algorithms
Python
1
star
33

energy-preserving-neural-network

When a data signal propagates through the Neural Network, it is not mandatory that the energy of the signal will be preserved throughout the neural computations. This research attempts at collecting (perhaps creating) techniques for preserving the Energy throughout the network.
Python
1
star