• Stars
    star
    151
  • Rank 244,631 (Top 5 %)
  • Language
    Python
  • License
    MIT License
  • Created over 2 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

[CVPR2023] DeepMapping2: Self-Supervised Large-Scale LiDAR Map Optimization

DeepMapping2: Self-Supervised Large-Scale LiDAR Map Optimization

Chao Chen*, Xinhao Liu*, Yiming Li, Li Ding, Chen Feng

News

[2023-06] πŸ”₯ Quaternion has been added as the default rotation representation, resulting in faster convergence from 30 epochs to 7 epochs.

[2023-06] πŸ”₯ Detailed instruction is available for implementing DeepMapping2 on a custom dataset.

[2023-03] The camera-ready version of our paper is available at arXiv. New figures are added to the supplementary material.

[2023-02] Our paper is accepted by CVPR 2023.

[2022-12] Our paper is available at arXiv and the project page is online.

Abstract

LiDAR mapping is important yet challenging in self-driving and mobile robotics. To tackle such a global point cloud registration problem, DeepMapping converts the complex map estimation into a self-supervised training of simple deep networks. Despite its broad convergence range on small datasets, DeepMapping still cannot produce satisfactory results on large-scale datasets with thousands of frames. This is due to the lack of loop closures and exact cross-frame point correspondences, and the slow convergence of its global localization network. We propose DeepMapping2 by adding two novel techniques to address these issues: (1) organization of training batch based on map topology from loop closing, and (2) self-supervised local-to-global point consistency loss leveraging pairwise registration. Our experiments and ablation studies on public datasets (KITTI, NCLT, and Nebula) demonstrate the effectiveness of our method. Our code will be released.

Getting Started:

Installation

The code is tested with Python 3.9, PyTorch 1.13.1, and CUDA 11.6.

To install the dependencies, you can create a virtual environment with

conda env create -f environment.yml

Data Preparation

To download the dataset used for training and testing, please refer to ./data/README.md

Usage

To train the model, execute the script

cd script/
./run_train.sh

The visualization and evaluation results will be saved in the results folder.

To use a different initial pose and pairwise registration, please edit INIT and PAIRWISE in the script to direct to the corresponding files.

Citation

If you find this work useful for your research, please cite our paper:

@inproceedings{chen2023deepmapping2,
  title={DeepMapping2: Self-Supervised Large-Scale LiDAR Map Optimization},
  author={Chen, Chao and Liu, Xinhao and Li, Yiming and Ding, Li and Feng, Chen},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={9306--9316},
  year={2023}
}

Related Project

DeepMapping (CVPR'2019 oral)

More Repositories

1

DeepMapping

[CVPR2019 Oral] Self-supervised Point Cloud Map Estimation
Python
193
star
2

SSCBench

SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving
Python
165
star
3

DiscoNet

[NeurIPS2021] Learning Distilled Collaboration Graph for Multi-Agent Perception
136
star
4

peac

[ICRA2014] Fast Plane Extraction Using Agglomerative Hierarchical Clustering (AHC)
C++
131
star
5

V2X-Sim

[RA-L2022] V2X-Sim Dataset and Benchmark
111
star
6

Occ4cast

Occ4cast: LiDAR-based 4D Occupancy Completion and Forecasting
Python
104
star
7

FLAT

[ICCV2021 Oral] Fooling LiDAR by Attacking GPS Trajectory
Python
67
star
8

SPARE3D

[CVPR2020] A Dataset for SPAtial REasoning on Three-View Line Drawings
Python
52
star
9

NYU-VPR

[IROS2021] NYU-VPR: Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences
Python
30
star
10

EgoPAT3D

[CVPR 2022] Egocentric Action Target Prediction in 3D
Jupyter Notebook
29
star
11

LLM4VPR

Can multimodal LLM help visual place recognition?
Python
28
star
12

Self-Supervised-SPARE3D

[CVPR 2022] Self-supervised Spatial Reasoning on Multi-View Line Drawings
Python
24
star
13

DeepExplorer

[RSS2023] Metric-Free Exploration for Topological Mapping by Task and Motion Imitation in Feature Space
Python
23
star
14

RealCity3D

Jupyter Notebook
22
star
15

DeepSoRo

[RA-L/ICRA2020] Real-time Soft Body 3D Proprioception via Deep Vision-based Sensing
Python
20
star
16

insta360_ros_driver

A ROS driver for Insta360 cameras, enabling real-time image capture, processing, and publishing in ROS environments.
Python
20
star
17

SNAC

[ICLR2023] Learning Simultaneous Navigation and Construction in Grid Worlds
Python
18
star
18

TF-VPR

Self-supervised place recognition by exploring temporal and feature neighborhoods
Python
15
star
19

DeepParticleRobot

[ICRA'22] A Deep Reinforcement Learning Environment for Particle Robot Navigation and Object Manipulation
Python
12
star
20

MARS

[CVPR2024] Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset
Python
11
star
21

pyAprilTag

python wrapper of AprilTag implemented in cv2cg, used for Robot Perception course
C++
10
star
22

BAGSFit

Primitive Fitting Using Deep Boundary Aware Geometric Segmentation
9
star
23

NYC-Indoor-VPR

Python
5
star
24

vis_nav_player

[ROB-GY 6203] Example Visual Navigation Player Code for Course Project
Python
5
star
25

Mobile3DPrinting

https://ai4ce.github.io/Mobile3DPrinting/
4
star
26

NYC-Event-VPR

Python
4
star
27

PointCloudSimulator

Code for simulating 2D point clouds
Python
3
star
28

M3DP-Sim

C++
2
star
29

pyP2Mesh

python wrapper for finding a point's closest point on a triangle mesh
Python
2
star
30

VIP_SelfDrive

Makefile
2
star
31

EgoPAT3Dv2

Python
2
star
32

LUWA

Jupyter Notebook
1
star
33

ai4ce_robot_ROS2_drivers

This repo contains all the ROS2 driver packages modified at AI4CE lab for working with various robots
1
star