• Stars
    star
    479
  • Rank 91,752 (Top 2 %)
  • Language
    C++
  • License
    GNU General Publi...
  • Created over 3 years ago
  • Updated 9 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Fast and Accurate Extrinsic Calibration for Multiple LiDARs and Cameras

Targetless Extrinsic Calibration of Multiple Small FoV LiDARs and Cameras using Adaptive Voxelization

Our journal paper is available on IEEE TIM. The pre-release code has been updated. Our experiment video is availalbe on YouTube and Bilibili. Please consider citing our paper if you find our code is useful.

@ARTICLE{9779777,
  author={Liu, Xiyuan and Yuan, Chongjian and Zhang, Fu},
  journal={IEEE Transactions on Instrumentation and Measurement},
  title={Targetless Extrinsic Calibration of Multiple Small FoV LiDARs and Cameras Using Adaptive Voxelization},
  year={2022},
  volume={71},
  number={},
  pages={1-12},
  doi={10.1109/TIM.2022.3176889}
}

Introduction

In this paper, we propose a fast, accurate, and targetless extrinsic calibration method for multiple LiDARs and cameras based on adaptive voxelization. On the theory level, we incorporate the LiDAR extrinsic calibration with the bundle adjustment method. We derive the second-order derivatives of the cost function w.r.t. the extrinsic parameter to accelerate the optimization. On the implementation level, we apply the adaptive voxelization to dynamically segment the LiDAR point cloud into voxels with non-identical sizes, and reduce the computation time in the process of feature correspondence matching.

Fig. 1 Dense colorized point cloud reconstructed with LiDAR poses and extrinsic calibrated with our proposed method.

Adaptive Voxelization

In both LiDAR-LiDAR and LiDAR-camera extrinsic calibration, we implement adaptive voxelization to accelerate the feature correspondence matching process. The point cloud map is dynamically segmented into voxels with non-identical sizes, such that only one plane feature is contained in each voxel. This process sufficiently saves the execution time of k-d tree searching from our previous work1 (see Fig. 2) and work2 (see Fig. 3).

Fig. 2 Adaptive voxelization in LiDAR-LiDAR extrinsic calibration.

Fig. 3 Adaptive voxelization in LiDAR-camera extrinsic calibration. A) real world image. B) raw point cloud of this scene. C) voxelization of previous work where the yellow circles indicate the false edge estimation. D) edges extracted with our proposed method.

1. Prerequisites

Our code has been tested on Ubuntu 16.04 with ROS Kinetic, Ubuntu 18.04 with ROS Melodic and Ubuntu 20.04 with ROS Noetic, Ceres Solver 1.14.x, OpenCV 3.4.14, Eigen 3.3.7, PCL 1.8.

2. Build and Run

Clone the repository and catkin_make:

cd ~/catkin_ws/src
git clone [email protected]:hku-mars/mlcc.git
cd .. && catkin_make
source ~/catkin_ws/devel/setup.bash

3. Run Our Example

The parameters base LiDAR (AVIA or MID), test scene (scene-1 or scene-2), adaptive_voxel_size, etc., could be modified in the corresponding launch file. We also provide the original rosbag files (scene-1 and scene-2) for your reference.

3.1 Multi-LiDAR Extrinsic Calibration

Step 1: base LiDAR pose optimization (the initial pose is stored in scene-x/original_pose)

roslaunch mlcc pose_refine.launch

Step 2: LiDAR extrinsic optimization (the initial extrinsic is stored in config/init_extrinsic)

roslaunch mlcc extrinsic_refine.launch

Step 3: pose and extrinsic joint optimization

roslaunch mlcc global_refine.launch

3.2 Multi-LiADR-Camera Extrinsic Calibration

roslaunch mlcc calib_camera.launch

3.3 Single LiDAR-Camera Calibration

We have added code for single LiDAR-camera extrinsic calibration using adaptive voxelization, which supports both pinhole and fisheye camera model. The FISHEYE macro is defined in calib_single_camera.hpp. You can try our provided fisheye camera data.

roslaunch mlcc calib_single_camera.launch

Fig. 4 Extrinsic calibration of fisheye camera and LiDAR in a single scene using adaptive voxelization. Left: distorted image. Right: colorized point cloud.

4. Run Your Own Data

To test on your own data, you need to save the LiDAR point cloud in .pcd format. Please only collect the point cloud and images when the LiDAR (sensor platform) is not moving for the optimal precision (or segment them from a complete rosbag). The base LiDAR poses and initial extrinsic values shall also be provided (in tx ty tz qw qx qy qz format). These initial values could be obtained by general SLAM and hand-eye calibration algorithms.

You may need to modify the parameters voxel_size (adaptive voxel size), feat_eigen_limit (feature eigen ratio) and downsmp_sz_base (downsampling size) for LiDAR-LiDAR extrinsic calibration to adjust the precision and speed. You need to change the corresponding path and topic name in the yaml files in the config folder.

5. Known Issues

Currently, we seperate the LiDAR extrinsic calibration process into three steps for debug reasons. In future release, we wish to combine them together to make it more convenient to use.

6. License

The source code is released under GPLv2 license.

We are still working on improving the performance and reliability of our codes. For any technical issues, please contact us via email [email protected] and [email protected].

For commercial use, please contact Dr. Fu Zhang [email protected].

More Repositories

1

FAST_LIO

A computationally efficient and robust LiDAR-inertial odometry (LIO) package
C++
2,549
star
2

r3live

A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package
C++
1,958
star
3

loam_livox

A robust LiDAR Odometry and Mapping (LOAM) package for Livox-LiDAR
C++
1,435
star
4

FAST-LIVO

A Fast and Tightly-coupled Sparse-Direct LiDAR-Inertial-Visual Odometry (LIVO).
C++
1,086
star
5

livox_camera_calib

This repository is used for automatic calibration between high resolution LiDAR and camera in targetless scenes.
C++
863
star
6

LiDAR_IMU_Init

[IROS2022] Robust Real-time LiDAR-inertial Initialization Method.
C++
834
star
7

Point-LIO

C++
745
star
8

r2live

R2LIVE: A Robust, Real-time, LiDAR-Inertial-Visual tightly-coupled state Estimator and mapping package
C++
721
star
9

BALM

An efficient and consistent bundle adjustment for lidar mapping
C++
700
star
10

ikd-Tree

This repository provides implementation of an incremental k-d tree for robotic applications.
C++
607
star
11

ImMesh

ImMesh: An Immediate LiDAR Localization and Meshing Framework
C++
590
star
12

STD

A 3D point cloud descriptor for place recognition
C++
548
star
13

VoxelMap

[RA-L 2022] An efficient and probabilistic adaptive voxel mapping method for LiDAR odometry
C++
479
star
14

FAST-LIVO2

FAST-LIVO2: Fast, Direct LiDAR-Inertial-Visual Odometry
471
star
15

HBA

[RAL 2023] A globally consistent LiDAR map optimization module
C++
437
star
16

IKFoM

A computationally efficient and convenient toolkit of iterated Kalman filter.
C++
420
star
17

M-detector

C++
362
star
18

LTAOM

C++
325
star
19

ROG-Map

C++
294
star
20

MARSIM

MARSIM: A light-weight point-realistic simulator for LiDAR-based UAVs
C++
283
star
21

D-Map

D-Map provides an efficient occupancy mapping approach for high-resolution LiDAR sensors.
C++
280
star
22

decentralized_loam

207
star
23

joint-lidar-camera-calib

Joint intrinsic and extrinsic LiDAR-camera calibration.
C++
194
star
24

SLAM-HKU-MaRS-LAB

In this repository, we present our research works of HKU-MaRS lab that related to SLAM
191
star
25

Voxel-SLAM

C++
185
star
26

Swarm-LIO2

Swarm-LIO2: Decentralized, Efficient LiDAR-inertial Odometry for UAV Swarms
158
star
27

dyn_small_obs_avoidance

C++
154
star
28

IPC

Integrated Planning and Control for Quadrotor Navigation in Presence of Sudden Crossing Objects and Disturbances
C++
147
star
29

btc_descriptor

137
star
30

PULSAR

C++
102
star
31

lidar_car_platfrom

48
star
32

iBTC

39
star
33

crossgap_il_rl

Python
38
star
34

multi_lidar_calib

28
star
35

Livox_handheld

25
star
36

mapping_eval

2
star