• Stars
    star
    267
  • Rank 153,579 (Top 4 %)
  • Language
    C++
  • License
    BSD 3-Clause "New...
  • Created over 2 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

🦊 Voxfield: non-Projective Signed Distance Fields for Online Planning and 3D Reconstruction [IROS' 22]

This repository represents the official implementation of the paper:

Voxfield: Non-Projective Signed Distance Fields for Online Planning and 3D Reconstruction

Yue Pan, Yves Kompis, Luca Bartolomei, Ruben Mascaro, Cyrill Stachniss, Margarita Chli

ETH Zurich | University of Bonn

1-min demo video | 7-min presentation | paper

Pipeline


Voxfield is an improved version of the volumetric mapping framework Voxblox on both the mapping accuracy and efficiency. Voxfield is heavily based on the original Voxblox implementation, with the additional capacity of a non-projective TSDF integration and an efficient ESDF integration based on TSDF map. The constructed TSDF and ESDF map can be used for 3D reconstruction and robot path planning in real-time.

This repository also provides the implementation of other state-of-the-art methods (Voxblox, FIESTA, EDT) on this task. A comparison of these methods' workflow are shown below.

Pipeline

Voxfield can be seamlessly integrated into those projects that originally use Voxblox as their volumetric mapping backbone (Cblox, Voxgraph, Kimera, etc). To make Voxfield directly compatible to those projects, we keep the name of our package as voxblox. In addition, we provide an example on a multi-resolution panoptic mapping framework Panmap for high-fidelity large-scale semantic reconstruction.

Installation

If you have installed ROS, set up the catkin workspace and the SSH key for github, you can use the following commands to install Voxfield:

cd ~/catkin_ws/src/
git clone [email protected]:VIS4ROB-lab/voxfield.git
wstool init . ./voxfield/voxfield_ssh.rosinstall     # If you created a new workspace
wstool merge -t . ./voxfield/voxfield_ssh.rosinstall # If you use an existing workspace
wstool update

And then compile Voxfield with:

cd ~/catkin_ws/src/
catkin build voxblox_ros

To avoid the potential conflict, if the original voxblox was installed, it's better to remove voxblox from catkin_ws/src.

For more details, please follow these instructions.

Instructions

  • To run the non-projective TSDF mapping and ESDF mapping of the proposed Voxfield, use the executables: np_tsdf_server and voxfield_server.
  • To run the original TSDF mapping and ESDF mapping of Voxblox, use the executables: tsdf_server and voxblox_server.
  • To run the ESDF mapping of FIESTA, use the executables: fiesta_server.
  • To run the ESDF mapping of EDT, use the executables: voxedt_server.
  • List of the ros services can be found here, which should be the same as Voxblox.

Example Usage

Run on the Cow & Lady real-world RGB-D dataset

  1. Download the dataset here or use the following command in a target folder:
wget http://robotics.ethz.ch/~asl-datasets/iros_2017_voxblox/data.bag
wget http://robotics.ethz.ch/~asl-datasets/iros_2017_voxblox/voxblox_cow_extras.zip
  1. Set the bag_file path in the launch files ./voxblox_ros/launch/voxfield_launch/cow_voxfield.launch to the path storing the Cow & Lady bag file.
  2. Run Voxfield mapping on the Cow & Lady dataset:
roslaunch voxblox_ros cow_voxfield.launch

Run on the KITTI real-world LiDAR or stereo dataset

  1. Download the full dataset here or a rosbag of sequence07 here. Then use the kitti_to_rosbag package to convert the full dataset to rosbags.
  2. Set the bag_file path in the launch files ./voxblox_ros/launch/voxfield_launch/kitti_voxfield.launch to the path storing the KITTI bag file.
  3. Run Voxfield mapping on the KITTI dataset:
roslaunch voxblox_ros kitti_voxfield.launch

Run on the MaiCity synthetic LiDAR dataset

  1. Download the dataset here or use the following command in a target folder:
wget https://www.ipb.uni-bonn.de/html/projects/mai_city/mai_city.tar.gz
tar -xvf mai_city.tar.gz
  1. Set the bag_file path in the launch files ./voxblox_ros/launch/voxfield_launch/mai_voxfield.launch to the path storing the MaiCity bag file.
  2. Run Voxfield mapping on the MaiCity dataset:
roslaunch voxblox_ros mai_voxfield.launch

Run on your own data (TBA)

Customizing, comparison and evaluation

To change the mapping and visualization parameters such as voxel size and truncation distance, please configure the .yaml files under ./voxblox_ros/cfg/param/ folder.

For the comparison with other state-of-the-art methods (Voxblox, FIESTA, EDT), set the bag_file path in the corresponding launch file [dataset]_[method].launch and launch it.

To evaluate the TSDF, mesh and ESDF mapping quality, one first need to use the ros service to save the corresponding map. You can configure the data path and evaluation setup here and conduct the evaluation by launching here. You may also check the evaluation metrics here.

Used for online path planning

Please check these instructions and the repository mav_voxblox_planning.

Replace Voxblox in your high-level volumetric mapping project

  • Panmap (Multi-resolution panoptic mapping)
  • Kimera (Semantic-metric mapping)
  • Voxgraph (Global consistent mapping)
  • ... ...

Citation

If you find this code useful for your work or use it in your project, please consider citing the paper:

@inproceedings{pan2022iros,
  title={Voxfield: Non-Projective Signed Distance Fields for Online Planning and 3D Reconstruction},
  author={Yue Pan and Yves Kompis and Luca Bartolomei and Ruben Mascaro and Cyrill Stachniss and Margarita Chli},
  booktitle={Proceedings of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS)},
  year={2022}
}

Acknowledgments

We thanks greatly for the authors of the following opensource projects:

  • Voxblox (underlying data structure, mesh reconstruction, visualization, comparison baseline)
  • FIESTA (comparison baseline)
  • VDB-EDT (comparison baseline)

More Repositories

1

ccm_slam

CCM-SLAM: Robust and Efficient Centralized Collaborative Monocular SLAM for Robotic Teams
C++
492
star
2

covins

COVINS-(G) -- A (Generic) Framework for Collaborative Visual-Inertial SLAM and Multi-Agent 3D Mapping
C++
384
star
3

HyperSLAM

Modular, open-source implementations of continuous-time simultaneous localization and mapping algorithms.
C++
229
star
4

multi_robot_coordination

"Multi-robot Coordination with Agent-Server Architecture for Autonomous Navigation in Partially Unknown Environments", IROS 2020
C++
91
star
5

voxfield-panmap

Multi-resolution panoptic volumetric mapingp using Voxfield as its mapping backbone [IROS' 22]
C++
86
star
6

aerial-depth-completion

Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation (RA-L/ICRA 2020)
Python
84
star
7

decoSLAM

C++
79
star
8

smb_path_planner

Repository for the path planner for SMB for the ETH Robotics Summer School
C++
51
star
9

mav_hardware

Documentation of the MAV hardware setup used at the Vision for Robotics Lab.
41
star
10

t_prm

Temporal Probabilistic Roadmap (T-PRM)
C++
36
star
11

HyperVariables

General variable and Lie group containers with extensive Eigen interoperability for non-linear optimizations.
C++
33
star
12

visensor_simulator

Simulator for ASL VI-Sensor using the RotorS simulator and Blender
Python
30
star
13

pose_graph_backend

Multi-agent Pose-Graph Sensor Fusion - from "Multi-robot Coordination with Agent-Server Architecture for Autonomous Navigation in Partially Unknown Environments", IROS 2020
C++
26
star
14

fast_multi_robot_exploration

"Fast Multi-UAV Decentralized Exploration of Forests", RAL 2023
C++
25
star
15

dmce

"Decentralised Multi-Robot Exploration using Monte Carlo Tree Search", IROS 2023
C++
25
star
16

place_recognition_dataset_icra2018

Outdoors datasets especially recorded for place recognition applications using both flying and hand-held setups.
20
star
17

HyperSensors

Implementations of common sensing modalities for computer vision.
C++
20
star
18

voxblox_multi_agent

Multi-agent version of Voxblox. Used in the paper "Multi-robot Coordination with Agent-Server Architecture for Autonomous Navigation in Partially Unknown Environments", IROS 2020
C++
20
star
19

mesh_based_mapping

Real-Time Mesh-based Scene Estimation for Aerial Inspection
19
star
20

robopt_open

Library to simplify implementation of optimization problems in Ceres. Open-source version.
C++
18
star
21

multirotors_landing

"Autonomous Emergency Landing for Multicopters using Deep Reinforcement Learning", IROS 2022
C++
17
star
22

vi_rpe

VI-RPE: Visual-Inertial Relative Pose Estimation for Aerial Vehicles
C++
12
star
23

HyperState

Collection of discrete- and continuous-time motion parametrizations.
C++
12
star
24

place_recognition_dataset_ral2019

Synthetic and real outdoor datasets especially recorded for place recognition applications using both flying and hand-held setups. Edit Manage topics
10
star
25

v4rl_mesh_based_mapping_lib

Real-Time Mesh-based Scene Estimation for Aerial Inspection - IROS2016
C++
10
star
26

vins_client_server

VINS-mono for client-server architecture.
C++
9
star
27

hyperion

Symbolic Continuous-Time Gaussian Belief Propagation Framework with Ceres Interoperability
6
star
28

v4rl-student-templates

ETHZ V4RL Student Templates
TeX
5
star
29

MemCDT

Domain-Adaptive Semantic Segmentation with Memory-Efficient Cross-Domain Transformers
Python
4
star
30

vi_bag_tools

Convert ros bagfiles with visual-inertial data to filesystem
Python
2
star
31

interday_crop_registration

Project Page for "Aerial Image-based Inter-day Registration for Precision Agriculture"- ICRA24
Jupyter Notebook
2
star
32

datasets-for-viewpoint-tolerant-semantic-segmentation

This repo contains the link and dscription of the training and test datasets used in project "Viewpoint-tolerant Semantic Segmentation"
1
star
33

comm_msgs

Utility messages for multi-robot state estimation
CMake
1
star
34

pcl_fusion

Repository for fusing the information from multiple agents and sending it to voxblox_multi_agent nodes
C++
1
star