IN OPEN ALPHA 0.0.1!
Happy Pi Day! This project is now in open alpha testing. For the full story, as well as instructions and support resources, please see this video and its description: https://youtu.be/TeWYAbhgaiU
Enjoy!
PRE-RELEASE TO-DO LIST
- Python API
- Blender Bridge
- Save NeRF
- Load NeRF
- Multi-GPU
- Multi-NeRF
- Dataset Preparation Tools
- Downloadable Binaries
Wishlist:
- Drastically improve rendering quality / reduce training loss
INTRODUCTION
Hello NeRF enthusiasts! Here you will find my NeRF rendering and training library. The core principles of this NeRF method are based on the incredible work of Thomas Müller, Alex Evans, Christoph Schied, and Alex Keller, in their paper Instant neural graphics primitives with a multiresolution hash encoding.
Yes, I realize there is already a CUDA implementation, but I wanted to take a crack at reimplementing this myself for the challenge, and for artistic uses such as:
- Spatial distortions
- Multiple NeRFs in one scene
- Multi-GPU capabilities
- Shadertoy-style effects
- Fractals
Since everything here has been written from scratch*, this codebase is permissively licensed and commercial-use-friendly.
(*with generous help from NeRF, nerfstudio, NerfAcc, and completely built on the tiny-cuda-nn backend.)
Enjoy!
-James
https://twitter.com/jperldev
INSTALLATION
Required toolkits:
Tested Configuration:
Build steps:
git clone [email protected]:JamesPerlman/TurboNeRF --recursive
cd TurboNeRF
cmake . -B build
cmake --build build -j
Until we have an extensible data loader, the test data I'm working with is here:
https://www.dropbox.com/sh/qkt4t1tk1o7pdc6/AAD218LLtAavRZykYl33mO8ia?dl=1
This project has only been tested on that one Lego scene. Real scenes appear to be broken for now.
LINUX ADDITIONAL STEP
There is an open issue when using CUDA 12 and PyBind11 (the latter is used by TurboNeRF for the Python module). Currently, a patch manually needs to be applied after checking out the TurboNeRF repository as shown above:
git clone [email protected]:JamesPerlman/TurboNeRF --recursive
cd TurboNeRF
# Apply patch
patch -p1 < ../patches/pybind11-cuda12.patch
# Build as usual
cmake . -B build
cmake --build build -j
THANK YOU
Extreme gratitude to open source projects that will allow this project to reach its full potential (in order of integration date):
- CUDA CMake Starter
- tiny-cuda-nn
- JSON for modern C++
- stb
- NeRF
- nerfstudio
- NerfAcc
- torch-ngp
- Nerfies
- glfw
- pybind11
- RenderMan
- glad
- cuda-cmake-github-actions
- pure-torch-ngp
CITATIONS
Next-level respect to the researchers much of this codebase is based off. Thank you for making your research public. This would not have been possible without you.
Mildenhall, Ben, et al. "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis." arXiv, 2020. doi:10.48550/arxiv.2003.08934 - (https://arxiv.org/abs/2003.08934)
Müller, Thomas, et al. "Instant neural graphics primitives with a multiresolution hash encoding." ACM Trans. Graph., 41(4), 102:1-102:15 - https://doi.org/10.1145/3528223.3530127
Max, Nelson. "Optical Models for Direct Volume Rendering." IEEE Transactions on Visualization and Computer Graphics (1995) - https://courses.cs.duke.edu/spring03/cps296.8/papers/max95opticalModelsForDirectVolumeRendering.pdf
Müller, T. (2021). tiny-cuda-nn (Version 1.7) [Computer software]. https://github.com/NVlabs/tiny-cuda-nn
Fawzi, A., Balog, M., Huang, A. et al. Discovering faster matrix multiplication algorithms with reinforcement learning. Nature 610, 47–53 (2022). https://doi.org/10.1038/s41586-022-05172-4
Alman, Josh, and Virginia Vassilevska Williams. "A Refined Laser Method and Faster Matrix Multiplication." arXiv, 2020, doi:10.48550/arxiv.2010.05846. https://arxiv.org/abs/2010.05846
Sabour, Sara, et al. "RobustNeRF: Ignoring Distractors with Robust Losses." arXiv, 2023, arXiv:2302.00833. https://arxiv.org/abs/2302.00833
SUPPORTERS
Extreme thank yous to these subscribers on Twitch (https://twitch.tv/jperldev) who support this project's development!
madclawgonzo - Requested a haiku written by ChatGPT: "Madclawgonzo / Subscribing to your stream / Software project."
anonymous - Requested to remain anonymous
gusround - https://github.com/candidogustavo
slowcon - "uncle slowcon is here with the 4090"
likid_3 - <3
cognitrol - Supporting cool work that helps the community explore technology
dankmatrix - (pending message)
seferidis - (pending message)
memepp - (pending message)
Dakren12 - (pending message)
Relakin - (Confused)
flouwr - (pending message)