• Stars
    star
    197
  • Rank 197,722 (Top 4 %)
  • Language
    C++
  • License
    GNU General Publi...
  • Created over 1 year ago
  • Updated 6 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A monocular plane-aided visual-inertial odometry

Monocular Visual-Inertial Odometry with Planar Regularities

ROS 1 Workflow

This is a real-time monocular visual-inertial odometry (VIO) system leverage environmental planes within a multi-state constraint Kalman filter (MSCKF) framework. At the core of our method is an efficient robust monocular-based plane detection algorithm which does not require additional sensing modalities such as a stereo, depth camera or neural network. The project is built on top of OpenVINS and thus shares similar tooling and architecture. This plane detection and tracking algorithm enables real-time regularization of point features to environmental planes which are either maintained in the state vector as long-lived planes, or marginalized for efficiency. Planar regularities are applied to both in-state SLAM and out-of-state MSCKF point features, enabling long-term point-to-plane loop-closures due to the large spacial volume of planes.

Installation and Building

You will need to ensure you have installed OpenCV 3 or 4, Eigen3, Ceres Solver, and a valid ROS 1 installation. Please additionally refer to the OpenVINS Installation Guide as dependencies are shared with it. For Ubuntu linux-based system the system dependencies are (in addition to ROS 1):

sudo apt-get install libeigen3-dev libboost-all-dev libceres-dev

We now setup a catkin workspace with both OpenVINS and the ov_plane project. Specifically, we can clone both projects:

mkdir -p ~/workspace/catkin_ws_plane/src/
cd ~/workspace/catkin_ws_plane/src/
git clone https://github.com/rpng/open_vins/
git clone https://github.com/rpng/ov_plane/ 
cd open_vins/
git checkout 74a63cf758f595cc280f2165e5a6576e7ef8521d # last tested OpenVINS commit
cd ~/workspace/catkin_ws_plane/
catkin build
source devel/setup.bash 

Running - Simulation

We extended the OpenVINS simulator to simulated environmental planes which contain environmental features. A certain number of points are simulated as being planar and "free" non-planar per frame. This simulator leverages a C++ implementation of our lips simulator.

roslaunch ov_plane simulation.launch

To generate simulation results, one inspect the results/run_simulation.sh script to reproduce the results of the paper. To evaluate the results relative to the groundtruth please refer to ov_eval of OpenVINS. The code has changed since publication to upgrade to the newest OpenVINS, but the results should still be representative.

Running - Real-World Datasets

We support both the The EuRoC MAV Dataset and our released RPNG AR Table Dataset datasets. To run on either real-world dataset, use one of the following commands:

roslaunch ov_plane serial.launch config:=euroc_mav dataset:=V1_01_easy bag:=<path_to_bag>
roslaunch ov_plane serial.launch config:=rpng_plane dataset:=table_01  bag:=<path_to_bag>

Users can reproduce the results presented in our publication with the scripts provided in the results/ folder. We additionally provide the script used to compute the evaluation, which are simple calls to ov_eval of OpenVINS. The groundtruth "truths" files can be found in ov_data. Please note the project has been updated since the time of publication and thus will cause slight variations in performance relative to the paper

./src/ov_plane/results/run_ros_eth.sh        # Euroc MAV
./src/ov_plane/results/run_ros_rpng_plane.sh # RPNG Indoor AR Table
./src/ov_plane/results/run_ros_tracking.sh   # Tracking statistics

Acknowledgements

This project additionally leveraged the following wonderful libraries.

  • CDT - For generating Delaunay triangulations
  • ikd-tree - For KNN searching without requiring ICP
  • tinyrender - For loading models for the AR demo

Credit / Licensing

This code was written by the Robot Perception and Navigation Group (RPNG) at the University of Delaware. If you have any issues with the code please open an issue on our github page with relevant implementation details and references. For researchers that have leveraged or compared to this work, please cite the following:

@Conference{Chen2023ICRA,
  Title      = {Monocular Visual-Inertial Odometry with Planar Regularities},
  Author     = {Chuchu Chen and Patrick Geneva and Yuxiang Peng and Woosik Lee and Guoquan Huang},
  Booktitle  = {Proc. of the IEEE International Conference on Robotics and Automation},
  Year       = {2023},
  Address    = {London, UK},
  Url        = {\url{https://github.com/rpng/ov_plane}}
}
@Conference{Geneva2020ICRA,
  Title      = {{OpenVINS}: A Research Platform for Visual-Inertial Estimation},
  Author     = {Patrick Geneva and Kevin Eckenhoff and Woosik Lee and Yulin Yang and Guoquan Huang},
  Booktitle  = {Proc. of the IEEE International Conference on Robotics and Automation},
  Year       = {2020},
  Address    = {Paris, France},
  Url        = {\url{https://github.com/rpng/open_vins}}
}

The codebase and documentation is licensed under the GNU General Public License v3 (GPL-3). You must preserve the copyright and license notices in your derivative work and make available the complete source code with modifications under the same license (see this; this is not legal advice).

More Repositories

1

open_vins

An open source platform for visual-inertial navigation research.
C++
2,111
star
2

R-VIO

Robocentric Visual-Inertial Odometry
C++
737
star
3

kalibr_allan

IMU Allan standard deviation charts for use with Kalibr and inertial kalman filters.
MATLAB
577
star
4

MINS

An efficient and robust multisensor-aided inertial navigation system with online calibration that is capable of fusing IMU, camera, LiDAR, GPS/GNSS, and wheel sensors. Use cases: VINS/VIO, GPS-INS, LINS/LIO, multi-sensor fusion for localization and mapping (SLAM). This repository also provides multi-sensor simulation and data.
C++
427
star
5

cpi

Closed-form Preintegration for Graph-based Visual-Inertial Navigation
C++
249
star
6

R-VIO2

Square-Root Robocentric Visual-Inertial Odometry with Online Spatiotemporal Calibration
C++
228
star
7

calc

Convolutional Autoencoder for Loop Closure
Python
190
star
8

lips

LiDAR-Inertial 3D Plane Simulator
MATLAB
150
star
9

suo_slam

Symmetry and Uncertainty-Aware Object SLAM for 6DoF Object Pose Estimation
Python
133
star
10

android-camera-calibration

Updated (opencv3 and camera2 API) android camera calibration application
C++
118
star
11

calc2.0

CALC2.0: Combining Appearance, Semantic and Geometric Information for Robust and Efficient Visual Loop Closure
Python
95
star
12

vicon2gt

Vicon-IMU fusion for groundtruth trajectory generation.
C++
95
star
13

ov_maplab

Interface for OpenVINS with the maplab project
C++
82
star
14

ocekf-slam

Observability-Constrained (OC)-EKF for 2D SLAM
MATLAB
79
star
15

android-dataset-recorder

Dataset collection app that will collect both IMG and IMU measurements for offline processing
Java
60
star
16

icalib.github.io

Inertial Aided Multi-Sensor Calibration
60
star
17

ov_secondary

Secondary posegraph adapted for interfacing with OpenVINS, based on VINS-Mono / VINS-Fusion.
C++
59
star
18

ar_table_dataset

Small-scale indoor table AR visual-inertial datasets with 6DoF groundtruth.
Python
53
star
19

reach_ros_node

ROS driver for the Reach RTK GNSS module by Emlid
Python
36
star
20

mast_project

Underwater Camera and Sonar SLAM (Kevin and Linde's MAST class project)
C++
36
star
21

pointgrey_ladybug

ROS Driver for Pointgrey Ladybug Cameras
C++
30
star
22

clatt

cooperative localization and target tracking
MATLAB
28
star
23

android_sensors_driver

ROS Driver for Android Sensors (opencv3 and camera1 API)
Java
26
star
24

xsens_standalone

Python Standalone library for use with the xsens IMU
Python
19
star
25

rosbags

Github mirror of https://gitlab.com/ternaris/rosbags
Python
18
star
26

kitti_parser

C++ parser for the RAW KITTI dataset, with callbacks
C++
15
star
27

gps_path_pub

Handy publishing of a path and and frame from a single GPS sensor.
C++
13
star
28

CSO

Calibration the rigid transformation between the stereo and odometry
C++
12
star
29

img_imu_record

Easy recording of image and imu data to disk
C++
9
star
30

orb_slam_mapmerge

A Versatile and Accurate Monocular SLAM (with map merging)
C++
7
star
31

apriltags-cpp

A simple ros wrapper for apriltag-cpp
C++
7
star
32

vins_source

Helper package with launch files for getting imu and video sources.
CMake
6
star
33

microstrain_comm

IMU driver for the Microstrain 3DM-GX3®-25. Converted to run on the ROS framework.
C
5
star
34

Microstrain-3DM-GX3-35

ROS driver for Microstrain 3DM-GX3-35 IMU
C
3
star
35

firefly

AscTec Firefly Documentation
1
star
36

ZED_Odom_Grabber

Grab zed stereo images and odometry from turtlebot at the same time.
C++
1
star
37

dvs128-viewer

A driver for dvs128 camera
Java
1
star