R2LIVE

A Robust, Real-time, LiDAR-Inertial-Visual tightly-coupled state Estimator and mapping

Our preprint paper: we have corrected some typos and errors of our previous version of paper, the amended paper can be access at here. When amending our paper, I would like to thanks narutojxl (焦小亮), who has found my errors and provided his corrections.

Our related video: our related video is now available on YouTube (click below images to open):

R2LIVE is a robust, real-time tightly-coupled multi-sensor fusion framework, which fuses the measurement from the LiDAR, inertial sensor, visual camera to achieve robust, accurate state estimation. Taking advantage of measurement from all individual sensors, our algorithm is robust enough to various visual failure, LiDAR-degenerated scenarios, and is able to run in real time on an on-board computation platform, as shown by extensive experiments conducted in indoor, outdoor, and mixed environment of different scale.

The reconstructed 3D maps of HKU main building are shown in (d), and the detail point cloud with the correspondence panorama images are shown in (a) and (b). (c) shows that our algorithm can close the loop by itself (returning the starting point) without any additional processing (e.g. loop closure). In (e), we merge our map with the satellite image to further examine the accuracy of our system.

1. Prerequisites

1.1 Ubuntu and ROS

Ubuntu 64-bit 16.04 or 18.04. ROS Kinetic or Melodic. ROS Installation and its additional ROS pacakge:

    sudo apt-get install ros-XXX-cv-bridge ros-XXX-tf ros-XXX-message-filters ros-XXX-image-transport

NOTICE: remember to replace "XXX" on above command as your ROS distributions, for example, if your use ROS-kinetic, the command should be:

    sudo apt-get install ros-kinetic-cv-bridge ros-kinetic-tf ros-kinetic-message-filters ros-kinetic-image-transport

1.2. Ceres Solver

Follow Ceres Installation.

1.3. livox_ros_driver

Follow livox_ros_driver Installation.

2. Build r2live on ROS:

Clone the repository and catkin_make:

    cd ~/catkin_ws/src
    git clone https://github.com/hku-mars/r2live.git
    cd ../
    catkin_make
    source ~/catkin_ws/devel/setup.bash

3. Docker Support (optional)

To further facilitate the building process, we add docker in our code. Docker environment is like a sandbox, thus makes our code environment-independent.

Docker build

    docker/build

Run

    xhost +
    docker/run

Run with gpu support

    xhost +
    docker/run_gpu

4. Run our examples

Download Our recorded rosbag and then

roslaunch r2live demo.launch
rosbag play YOUR_DOWNLOADED.bag

If everything is correct, you will get the result that matches our paper:) The result of indoor_aggressive.bag (the Experiment-1 in our paper):

and hku_main_building.bag (our Experiment-3).

5. Sampling and run your own data

Since the LiDAR data and IMU data published by the official Livox-ros-driver is with the timestamp of LiDAR (started from 0 in each recording), and the timestamp of the image is usually recorded with the timestamp of the operation system. To make them working under the same time-based, we modified the source code of Livox-ros-driver, which is available at here. We suggest you replace the official driver with it when sampling your own data for R2LIVE.

6. Acknowledgments

Our repository contains two main subsystems, with our LiDAR-inertial and visual-inertial system is developed based on FAST-LIO and VINS-Mono, respectively. Besides, our implementations also use the codes of ikd-Tree, BALM and loam-livox.

7. License

The source code is released under GPLv2 license.

We are still working on improving the performance and reliability of our codes. For any technical issues, please contact me via email Jiarong Lin < [email protected] >.

For commercial use, please contact Dr. Fu Zhang < [email protected] >