• Stars
    star
    543
  • Rank 81,848 (Top 2 %)
  • Language
    C++
  • License
    GNU General Publi...
  • Created over 10 years ago
  • Updated over 8 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Volumetric 3D Mapping in Real-Time on a CPU

fastfusion

Volumetric 3D Mapping in Real-Time on a CPU

This code implements the approach for real-time 3D mapping on a CPU as described in the following research paper:

http://vision.in.tum.de/_media/spezial/bib/steinbruecker_etal_icra2014.pdf

Volumetric 3D Mapping in Real-Time on a CPU (F. Steinbruecker, J. Sturm, D. Cremers), In Int. Conf. on Robotics and Automation, 2014.

alt tag

Demo video: http://youtu.be/7s9JePSln-M

@string{icra="Int. Conf. on Robotics and Automation"}
@inproceedings{Steinbruecker-etal-icra14,
  author = {F. Steinbruecker and J. Sturm and D. Cremers},
  title = {Volumetric 3D Mapping in Real-Time on a CPU},
  booktitle = icra,
  year = {2014},
  address = {Hongkong, China},
  titleurl = {steinbruecker_etal_icra2014.pdf},
  topic = {3D Reconstruction},
  keywords =  {RGB-D,Fusion,3d-reconstruction}
}

Installation

$ git clone https://github.com/tum-vision/fastfusion.git`

$ cd fastfusion

$ cmake .

$ make

Preparation of the data

The software takes a text file as input which contains per file

  • the camera pose
  • the depth image filename
  • the color image filename

You can either generate such a file yourself (e.g., by running Christan Kerl's DVO SLAM:

http://vision.in.tum.de/data/software/dvo

available as open source on our homepage) or you can download sequences from the TUM RGB-D benchmark:

http://vision.in.tum.de/data/datasets/rgbd-dataset/

For simplicity, we take a pre-recorded sequence from the TUM RGB-D benchmark.

$ mkdir ~/data

$ cd ~/data

$ wget http://vision.in.tum.de/rgbd/dataset/freiburg3/rgbd_dataset_freiburg3_long_office_household.tgz

$ tar xvzf rgbd_dataset_freiburg3_long_office_household.tgz

Now we need to generate the text file. For this, we use the associate.py tool from the RGB-D benchmark website. We need to run it twice, as we join the camera poses, the depth image list and the color image list into a single file:

$ cd ~/fastfusion/

$ ./associate.py ~/data/rgbd_dataset_freiburg3_long_office_household/groundtruth.txt ~/data/rgbd_dataset_freiburg3_long_office_household/depth.txt > tmp.txt

$ ./associate.py tmp.txt ~/data/rgbd_dataset_freiburg3_long_office_household/rgb.txt > ~/data/rgbd_dataset_freiburg3_long_office_household/associate.txt

The resulting text file should look as follows:

$ head ~/data/rgbd_dataset_freiburg3_long_office_household/associate.txt
1341847980.790000 -0.6832 2.6909 1.7373 0.0003 0.8617 -0.5072 -0.0145 1341847980.786879 depth/1341847980.786879.png 1341847980.786856 rgb/1341847980.786856.png
1341847980.820100 -0.6821 2.6914 1.7371 0.0003 0.8609 -0.5085 -0.0151 1341847980.822989 depth/1341847980.822989.png 1341847980.822978 rgb/1341847980.822978.png
1341847980.850000 -0.6811 2.6918 1.7371 0.0001 0.8610 -0.5084 -0.0159 1341847980.854690 depth/1341847980.854690.png 1341847980.854676 rgb/1341847980.854676.png
[..]

Running the code

$ ./bin/onlinefusion ~/data/rgbd_dataset_freiburg3_long_office_household/associate.txt --thread-fusion

After some debugging output on the console, a window with a 3D viewer should open. To start the reconstruction process, press "S".

If you run the program for the first time, press and hold the CTRL key and turn your scroll wheel. This is only needed once to "free" the camera viewpoint. After this, you can pan (right click) and rotate (left click) the view as you wish using your mouse.

Further options

   ./bin/onlinefusion  [--intrinsics <string>] [--imagescale <float>]
                       [--threshold <float>] [--scale <float>]
                       [--max-camera-distance <float>]
                       [--consistency-checks <int>] [-k <int>] [-e <int>]
                       [-s <int>] [--incremental-meshing] [-c] [-b] [-v]
                       [--thread-image] [--thread-fusion]
                       [--thread-meshing] [-l <string>] [--] [--version]
                       [-h] <string> ...


Where: 

   --intrinsics <string>
     File with Camera Matrix

   --imagescale <float>
     Image Depth Scale

   --threshold <float>
     Threshold

   --scale <float>
     Size of the Voxel

   --max-camera-distance <float>
     Maximum Camera Distance to Surface

   --consistency-checks <int>
     Number of Depth Consistency Checks

   -k <int>,  --imagestep <int>
     Use every kth step

   -e <int>,  --endimage <int>
     Number of the End Image

   -s <int>,  --startimage <int>
     Number of the Start Image

   --incremental-meshing
     Perform incremental Meshing

   -c,  --loopclosures
     Read Multiple Trajectories and perform Loop Closures

   -b,  --buffer
     Buffer all Images

   -v,  --viewer
     Show a Viewer after Fusion

   --thread-image
     Thread reading the Images from Hard Disk

   --thread-fusion
     Thread the Fusion inside the Viewer

   --thread-meshing
     Thread the Meshing inside the Fusion

   -l <string>,  --loadmesh <string>
     Loads this mesh

   --,  --ignore_rest
     Ignores the rest of the labeled arguments following this flag.

   --version
     Displays version information and exits.

   -h,  --help
     Displays usage information and exits.

   <string>  (accepted multiple times)
     The File Names

alt tag

More Repositories

1

lsd_slam

LSD-SLAM
C++
2,486
star
2

tandem

[CoRL 21'] TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo
C++
911
star
3

LDSO

DSO with SIM(3) pose graph optimization and loop closure
C++
653
star
4

dvo_slam

Dense Visual Odometry and SLAM
C++
607
star
5

online_photometric_calibration

Implementation of online photometric calibration (https://vision.in.tum.de/research/vslam/photometric-calibration)
C++
306
star
6

mono_dataset_code

Code for Monocular Visual Odometry Dataset - https://vision.cs.tum.edu/data/datasets/mono-dataset
C++
261
star
7

tum_ardrone

Repository for the tum_ardrone ROS package, implementing autonomous flight with PTAM-based visual navigation for the Parrot AR.Drone.
C++
221
star
8

dvo

Dense Visual Odometry
C++
148
star
9

fusenet

This repository is the official release of the code for the following paper "FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-based CNN Architecture" which is published at the 13th Asian Conference on Computer Vision (ACCV 2016).
C++
126
star
10

pnec

[CVPR 2022] README.md The Probabilistic Normal Epipolar Constraint for Frame-To-Frame Rotation Optimization under Uncertain Feature Positions
C++
117
star
11

captcha_recognition

Python
71
star
12

intrinsic-neural-fields

[ECCV '22] Intrinsic Neural Fields: Learning Functions on Manifolds
Jupyter Notebook
66
star
13

dbatk

Distributed Bundle Adjustment Toolkit
59
star
14

fastms

Real-Time Minimization of the Piecewise Smooth Mumford-Shah Functional
C++
57
star
15

ardrone_autonomy

This is a slightly modified version of the official ardrone_autonomy package, which You can find here: https://github.com/AutonomyLab/ardrone_autonomy
C
53
star
16

learn_prox_ops

Implementation of "Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems"
Python
43
star
17

tum_simulator

C++
40
star
18

prost

A fast and flexible convex optimization framework based on proximal splitting
C++
35
star
19

afs

Automatic Feature Selection
C++
31
star
20

rgbd_scribble_benchmark

RGB-D Scribble-based Segmentation Benchmark
Python
26
star
21

autonavx_ardrone

Code for AR.Drone Exercises
C++
24
star
22

autonavx_web

interactive exercises for AUTONAVx course
JavaScript
24
star
23

sublabel_relax

Code for sublabel-accurate multi-labeling papers (published at CVPR '16, ECCV '16)
C++
20
star
24

csd_lmnn

Combined Spectral Descriptors and LMNN for non-rigid 3D shape retrieval
MATLAB
19
star
25

rgbd_demo

Simple ROS demo for processing RGB-D data
C++
17
star
26

mem

Masked Event Modeling: Self-Supervised Pretraining for Event Cameras (WACV '24)
Python
15
star
27

kfusion_ros

ROS integration for kfusion
C++
11
star
28

openni2_camera

OpenNI2 camera node for ROS
C++
9
star
29

articulation

articulation models
C++
6
star
30

nnascg

Source code for experiments in paper "Deriving Neural Network Design and Learning from the Probabilistic Framework of Chain Graphs" by Yuesong Shen and Daniel Cremers.
Python
4
star
31

lgm

Implementation of Layered Graphical Model with demo code
Python
4
star
32

dca

Source code for the NeurIPS 2022 paper "Deep Combinatorial Aggregation"
Python
4
star
33

flbo

2
star
34

hierahyp

1
star