TurtleZhong/LVIO-SAM TurtleZhong/LVIO-SAM

  • Stars
    star
    173
  • Rank 220,124 (Top 5 %)
  • Language
    C++
  • Created over 3 years ago
  • Updated over 2 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
TurtleZhong/LVIO-SAM
github

TurtleZhong

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A Multi-sensor Fusion Odometry via Smoothing and Mapping.

LVIO-SAM

A multi-sensor fusion odometry, LVIO-SAM, which fuses LiDAR, stereo camera and inertial measurement unit (IMU) via smoothing and mapping.

  • The code is still being integrated. we will release it in the feature.

LVIO-SAM

A multi-sensor fusion odometry, LVIO-SAM, which fuses LiDAR, stereo camera and inertial measurement unit (IMU) via smoothing and mapping.

Contributors Forks Stargazers Issues


Logo Logo

LVIO-SAM

A multi-sensor fusion odometry, LVIO-SAM, which fuses LiDAR, stereo camera and inertial measurement unit (IMU) via smoothing and mapping.
Demo Youtube · Demo Bilibili · Report Bug · Request Feature

Table of Contents
  1. About The Project
  2. Simulations environment
  3. How to run
  4. Roadmap
  5. Contributing
  6. License
  7. Contact
  8. Acknowledgements

About The Project

   This project is provide a a multi-sensor fusion odometry, LVIO-SAM, which fuses LiDAR,stereo camera and inertial measurement unit (IMU) via smoothing and mapping.

!!!Important Notes!!!

Simulations environment

   We modify the Gazebo world proposed in here and adding our own sensors to test our proposed method. We use Husky as the base robot and we modify the urdf. The robot is equiped with A velodyne VLP 16 lidar, stereo camera(640x480) and an IMU (50Hz).

Download the CMU campus model to sim_env/husky_gazebo/mesh/

cd YOUR_WORD_PATH/LVIO_SAM/sim_env/husky_gazebo/mesh/
unzip autonomus_exploration_environments.zip

I guess c campus model it to ~/.gazebo/models/.

cd autonomus_exploration_environments/
cp -r campus ~/.gazebo/models/

you can launch gazebo and find campu model to check if it is OK.

git clone https://github.com/TurtleZhong/LVIO-SAM.git

cd YOUR_PATH/LVIO-SAM
catkin build -DCMAKE_BUILD_TYPE=Release
source devel/setup.bash

roslaunch husky_gazebo husky_campus.launch

It will take a few minutes to load the world. please start a new terminal and launch the husky and sensor model.

roslaunch husky_gazebo spawn_husky.launch

If everything is OK, you will get this:

[Logo]

if you want control the robot, you can use the keyboard i,j,k,l etc.

rosrun teleop_twist_keyboard teleop_twist_keyboard.py

How to run in Docker

   Since our code is still being integrated. we will release it in the feature. But we provide a docker environment for users. So Docker should be correctly installed.

  Step1. Prepare Datasets

  1. KITTI datasets
wget https://s3.eu-central-1.amazonaws.com/avg-kitti/raw_data/2011_09_30_drive_0027/2011_09_30_drive_0027_sync.zip
wget https://s3.eu-central-1.amazonaws.com/avg-kitti/raw_data/2011_09_30_drive_0027/2011_09_30_drive_0027_extract.zip
wget https://s3.eu-central-1.amazonaws.com/avg-kitti/raw_data/2011_09_30_calib.zip
unzip 2011_09_30_drive_0084_sync.zip
unzip 2011_09_30_drive_0084_extract.zip
unzip 2011_09_30_calib.zip
python kitti2bag.py -t 2011_09_30 -r 0027 raw_synced .

That's it. You have a bag that contains your data.

╰─$ rosbag info kitti_2011_09_30_drive_0027_synced.bag 
path:        kitti_2011_09_30_drive_0027_synced.bag
version:     2.0
duration:    1:55s (115s)
start:       Sep 30 2011 12:40:25.07 (1317357625.07)
end:         Sep 30 2011 12:42:20.41 (1317357740.41)
size:        6.0 GB
messages:    35278
compression: none [4435/4435 chunks]
types:       geometry_msgs/TwistStamped [98d34b0043a2093cf9d9345ab6eef12e]
             sensor_msgs/CameraInfo     [c9a58c1b0b154e0e6da7578cb991d214]
             sensor_msgs/Image          [060021388200f6f0f447d0fcd9c64743]
             sensor_msgs/Imu            [6a62c6daae103f4ff57a132d6f95cec2]
             sensor_msgs/NavSatFix      [2d3a8cd499b9b4a0249fb98fd05cfa48]
             sensor_msgs/PointCloud2    [1158d486dd51d683ce2f1be655c3c181]
topics:      /gps/fix                                 1106 msgs    : sensor_msgs/NavSatFix     
             /gps/vel                                 1106 msgs    : geometry_msgs/TwistStamped
             /imu_correct                            11556 msgs    : sensor_msgs/Imu           
             /imu_raw                                11556 msgs    : sensor_msgs/Imu           
             /kitti/camera_color_left/camera_info     1106 msgs    : sensor_msgs/CameraInfo    
             /kitti/camera_color_left/image_raw       1106 msgs    : sensor_msgs/Image         
             /kitti/camera_color_right/camera_info    1106 msgs    : sensor_msgs/CameraInfo    
             /kitti/camera_color_right/image_raw      1106 msgs    : sensor_msgs/Image         
             /kitti/camera_gray_left/camera_info      1106 msgs    : sensor_msgs/CameraInfo    
             /kitti/camera_gray_left/image_raw        1106 msgs    : sensor_msgs/Image         
             /kitti/camera_gray_right/camera_info     1106 msgs    : sensor_msgs/CameraInfo    
             /kitti/camera_gray_right/image_raw       1106 msgs    : sensor_msgs/Image         
             /points_raw                              1106 msgs    : sensor_msgs/PointCloud2

Other source files can be found at KITTI raw data page.

  1. sim_env datasets

You can record datasets from our simulation environments or download the sample dataset from BaiduYun Link, the extract code is f8to.

  Get docker images and create your own datasets..

docker pull xinliangzhong/ubuntu-18.04-novnc-lvio-sam:v1

use docker images check the image is ok.

docker run -it --rm -p 8080:80 xinliangzhong/ubuntu-18.04-novnc-lvio-sam:v1

then open the Chrome browser and type http://127.0.0.1:8080/

open 3 terminal and run

cd /root
source .bashrc
cd work/ws_lvio/
source devel/setup.bash

roslaunch husky_gazebo husky_campus.launch
roslaunch husky_gazebo husky_campus.launch

It will take a few minutes to load the world. please start a new terminal and launch the husky and sensor model.

roslaunch husky_gazebo spawn_husky.launch

  

roslaunch husky_viz view_robot.launch

If everything is OK, you will get this in your chrome browser:

[Logo]

Run LVIO-SAM in docker

Follow the above steps to get the docker image, and open it in browser:

[Logo] 

cd /root
source .bashrc
cd work/ws_lvio/
source devel/setup.bash

roslaunch lvio_sam run_kitti_debug_test_vo_between_factor.launch #for kitti dataset.
roslaunch lvio_sam run_kitti_debug_test_vo_between_factor.launch #for sim dataset.

we prepare 2 sample bag in the docker, you can use it directly.

rosbag play kitti_2011_09_30_drive_0027_synced.bag --pause --clock #for kitti dataset.
rosbag play 2021-08-04-09-49-56.bag --pause --clock #for sim dataset.

If everything is OK, you will get this in your chrome browser:

[Logo]

Roadmap

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

  1. Fork the Project
  2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
  3. Commit your Changes (git commit -m 'Add some AmazingFeature')
  4. Push to the Branch (git push origin feature/AmazingFeature)
  5. Open a Pull Request

License

Distributed under the MIT License.

Contact

Xinliang Zhong - @zxl - [email protected]

Project Link: https://github.com/TurtleZhong/LVIO-SAM

Citation

@inproceedings{zhong2021lvio,
  title={LVIO-SAM: A Multi-sensor Fusion Odometry via Smoothing and Mapping},
  author={Zhong, Xinliang and Li, Yuehua and Zhu, Shiqiang and Chen, Wenxuan and Li, Xiaoqian and Gu, Jason},
  booktitle={2021 IEEE International Conference on Robotics and Biomimetics (ROBIO)},
  pages={440--445},
  year={2021},
  organization={IEEE}
}

Acknowledgements

More Repositories

1

Map-based-Visual-Localization

A general framework for map-based visual localization. It contains 1) Map Generation which support traditional features or deeplearning features. 2) Hierarchical-Localizationvisual in visual(points or line) map. 3)Fusion framework with IMU, wheel odom and GPS sensors.
610
star
2

AVP-SLAM-SIM

A basic implementation(not official code) of AVP-SLAM(IROS 2020) in simulation. https://arxiv.org/abs/2007.01813
CMake
379
star
3

msckf_mono

VIO_Learning. A simple implement of mono msckf in https://github.com/TurtleZhong/msckf_mono/tree/master/src/msckf_mine_1.0
C++
309
star
4

camera_lidar_calibration_v2

ROS VERSION: A tool used for calibrate 2D laser range finder (LRF) and camera.version1：https://github.com/TurtleZhong/camera_lidar_calibration
Makefile
115
star
5

camera_lidar_calibration

A tool used for calibrate the extrinsic between 2D laser range finder (LRF) and camera. ROS Version: https://github.com/TurtleZhong/camera_lidar_calibration_v2
C++
72
star
6

PoseGraph-Ceres

An offline tool for pose-graph-optimization.
C++
69
star
7

Visual-Inertial-Synchronization-Hardware

This project aims to hardware-synchronize camera and IMU so that both use the same (millisecond precise) time base.
C++
65
star
8

hfnet_ros

A cpp or ROS Warpper of HF-NET inference part (https://github.com/ethz-asl/hfnet)
C++
57
star
9

ESKF-All-in-One-Open

Error State Kalman Filter All in One, from Theory to Practice.
54
star
10

avp-bev-open

A pkg stiching around view images(4-6cameras) to generate bird's eye view.
38
star
11

Projects-Summary

本人完成的项目汇总
23
star
12

vins_notes

vins-mono初始化部分详细教程
HTML
17
star
13

Gazebo-models-generator

A tool using to generate Gazebo models from 3D models.(Sketchup files, *.dae, *.obj, *.stl, etc.)
Python
16
star
14

ORB_SLAM2_error_analysis

ORB_SLAM2_error_analysis
C++
12
star
15

Particle_Filter_Demo

A demo for particle filter.
Python
9
star
16

msckf_notes

HTML
8
star
17

ros_projects

Some ros projects
C++
6
star
18

SLAM_Learning

projects after 2017
CMake
2
star
19

paper_reading_in_DM

some papers reading in DeepMotion
HTML
2
star
20

leetcode

leetcode exercises
C++
2
star
21

Super-Bible

Super Bible of SLAM
2
star
22

CarND-Motion-Prediction-Control-Project

C++
1
star
23

Udacity_SelfDrivingCar_2SensorFusion_Project2_ParticalFilter

C++
1
star