• When reconstructing a smooth object that does not have a high level of detail. The smooth mode works best for objects that have relatively simple, flowing surfaces without a lot of intricate features.

• When you want a higher-fidelity substitute for instant-nsr that takes a similar amount of time (within 20 minute) to generate but with fewer holes in the resulting model.

Information you need to know before you start:

• The smooth reconstruction mode's reliance on curvature loss can over-smooth geometry, failing to capture flat surface structures and subtle variations on flatter regions of the original object.
• This mode relies on sparse points generated by colmap to guide the geometry in the early stage of training. However, SFM (Structure from Motion) can sometimes generate noisy point clouds due to factors such as repeated texture, inaccurate poses, or incorrect point matches. To address these issues, one possible solution is to utilize more powerful SFM tools like hloc or DetectorFreeSfM. Additionally, post-processing techniques can be employed to further refine the point cloud. For example, using methods like Radius Outlier Removal in Open3D or pixsfm can help eliminate outliers and improve the quality of the point cloud.

Now it is time to start by running:

bash run_neuralangelo-colmap_sparse.sh ${INPUT_DIR}

This script is designed to automate the process of running SFM without the need for any preparation beforehand. It will automatically initiate the reconstruction process and export the resulting mesh. The output files will be saved in the logs directory.

If mask is available and placed at the right place under data_folder you could start by running:

bash run_neuralangelo-colmap_sparse.sh ${INPUT_DIR}_filtered

Generating high-fidelity surface reconstructions with only RGB inputs in 20,000 steps (around 20 minutes) is challenging, especially for sparse in-the-wild captures where occlusion and limited views make surface reconstruction an underconstrained problem. This can lead to optimization instability and difficulty converging. Introducing lidar, ToF depth, or predicted depth can help stabilize optimization and accelerate training. However, directly regularizing rendered depth is suboptimal due to bias introduced by density2sdf. Moreover, ensuring consistent depth across views is difficult, especially with lower-quality ToF sensors or predicted depth. We propose directly regularizing the SDF field using MVS point clouds and normals to alleviate the bias

Information you need to know before you start:

• An aligned dense point cloud with normal is necessary, you could specify the relative path at dataset.dense_pcd_path in the config file
• The point cloud could be generated from various methods, either from traditional MVS like colmap or OpenMVS, or learning-based MVS method. You could even generate the point cloud using commercial photogrammetry software like metashape and DJI.

Now it is time to start by running:

bash run_neuralangelo-colmap_dense.sh  ${INPUT_DIR}

Information you need to know before you start:

• We implement several key techniques from UniSDF including: 1. Camera and Reflected Radiance Compositions; 2. Coarse-to-fine training strategy However, we currently do not implement contraction and proposal sampling. Instead, we use NeRF++ for background modeling and occupancy grids for sampling acceleration.
• Due to GPU resource constraints, we do not implement the exact same parameters and training regimen as the UniSDF paper, which uses 8 V100s for 3 hours of training. In the future, we aim to continue experimenting with UniSDF to push reconstruction quality and efficiency. But for now, we have some adaptations allow UniSDF-enhanced performance on readily available GPU hardware with reasonable training times. Specifically, our modifications include:
  • Reduced MLP dimension from 256 to 64
  • Only 2 layers for the radiance MLPs
  • 16 level NGP grid with 2 channels per level (vs 4 channels)
• TODO List:
  • Contraction
  • Proposal Sampling
  • Preprocessed dataset links
  • Other regularizations in UniSDF

Now it is time to start by running:

bash run_neuralangelo-colmap_sparse_reflection.sh  ${INPUT_DIR}

1. Q: CUDA out of memory.

   A: Instant-angelo requires at least 10GB GPU memory. If you run out of memory, consider decreasing model.num_samples_per_ray from 1024 to 512

2. Q: What's the License for this repo?

   A: This repository is built on top of instant-nsr-pl and is licensed under the MIT License. The materials, code, and assets in this repository can be used for commercial purposes without explicit permission, in accordance with the terms of the MIT License. Users are free to use, modify, and distribute this content, even for commercial applications. However, appropriate attribution to the original instant-nsr-pl authors and this repository is requested. Please refer to the LICENSE file for full terms and conditions.

3. Q: The reconstruction of my custom dataset is bad.

   A: This repository is under active development and its robustness across diverse real-world data is still unproven. Users may encounter issues when applying the method to new datasets. Please open an issue for any problems or contact the author directly at [email protected].

4. Q: Generate dense prior with Vis-MVSNet is slow

   A: Currently, preprocessing takes around 10~15 minutes for 300 frames, but there still remains much room to improve efficiency by replacing Vis-MVSNet with state-of-the-art methods like MVSFormer or SimpleRecon. Moreover, preprocessing time could be substantially reduced by leveraging quantization and TensorRT. Overall, MVSNet enables the creation of the necessary point cloud much faster than traditional MVS approaches, by an order of magnitude.

5. Q: This project fails to run on Windows

   A: This project has not been tested on Windows and the scripts may have compatibility issues. For the best experience at this stage of development, we recommend running experiments on a Linux system. We apologize that Windows support cannot be guaranteed currently. Please feel free to open an issue detailing any problems encountered when attempting to run on Windows. Community feedback will help improve cross-platform compatibility going forward.