• Stars
    star
    156
  • Rank 239,589 (Top 5 %)
  • Language
    Python
  • License
    MIT License
  • Created over 2 years ago
  • Updated 5 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

[CVPR2023] DeepMapping2: Self-Supervised Large-Scale LiDAR Map Optimization

DeepMapping2: Self-Supervised Large-Scale LiDAR Map Optimization

Chao Chen*, Xinhao Liu*, Yiming Li, Li Ding, Chen Feng

News

[2023-06] πŸ”₯ Quaternion has been added as the default rotation representation, resulting in faster convergence from 30 epochs to 7 epochs.

[2023-06] πŸ”₯ Detailed instruction is available for implementing DeepMapping2 on a custom dataset.

[2023-03] The camera-ready version of our paper is available at arXiv. New figures are added to the supplementary material.

[2023-02] Our paper is accepted by CVPR 2023.

[2022-12] Our paper is available at arXiv and the project page is online.

Abstract

LiDAR mapping is important yet challenging in self-driving and mobile robotics. To tackle such a global point cloud registration problem, DeepMapping converts the complex map estimation into a self-supervised training of simple deep networks. Despite its broad convergence range on small datasets, DeepMapping still cannot produce satisfactory results on large-scale datasets with thousands of frames. This is due to the lack of loop closures and exact cross-frame point correspondences, and the slow convergence of its global localization network. We propose DeepMapping2 by adding two novel techniques to address these issues: (1) organization of training batch based on map topology from loop closing, and (2) self-supervised local-to-global point consistency loss leveraging pairwise registration. Our experiments and ablation studies on public datasets (KITTI, NCLT, and Nebula) demonstrate the effectiveness of our method. Our code will be released.

Getting Started:

Installation

The code is tested with Python 3.9, PyTorch 1.13.1, and CUDA 11.6.

To install the dependencies, you can create a virtual environment with

conda env create -f environment.yml

Data Preparation

To download the dataset used for training and testing, please refer to ./data/README.md

Usage

To train the model, execute the script

cd script/
./run_train.sh

The visualization and evaluation results will be saved in the results folder.

To use a different initial pose and pairwise registration, please edit INIT and PAIRWISE in the script to direct to the corresponding files.

Citation

If you find this work useful for your research, please cite our paper:

@inproceedings{chen2023deepmapping2,
  title={DeepMapping2: Self-Supervised Large-Scale LiDAR Map Optimization},
  author={Chen, Chao and Liu, Xinhao and Li, Yiming and Ding, Li and Feng, Chen},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={9306--9316},
  year={2023}
}

Related Project

DeepMapping (CVPR'2019 oral)

More Repositories

1

DeepMapping

[CVPR2019 Oral] Self-supervised Point Cloud Map Estimation
Python
193
star
2

SSCBench

SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving
Python
166
star
3

DiscoNet

[NeurIPS2021] Learning Distilled Collaboration Graph for Multi-Agent Perception
136
star
4

peac

[ICRA2014] Fast Plane Extraction Using Agglomerative Hierarchical Clustering (AHC)
C++
134
star
5

Occ4cast

Occ4cast: LiDAR-based 4D Occupancy Completion and Forecasting
Python
112
star
6

V2X-Sim

[RA-L2022] V2X-Sim Dataset and Benchmark
111
star
7

FLAT

[ICCV2021 Oral] Fooling LiDAR by Attacking GPS Trajectory
Python
67
star
8

SPARE3D

[CVPR2020] A Dataset for SPAtial REasoning on Three-View Line Drawings
Python
52
star
9

MARS

[CVPR2024] Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset
Python
36
star
10

NYU-VPR

[IROS2021] NYU-VPR: Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences
Python
31
star
11

EgoPAT3D

[CVPR 2022] Egocentric Action Target Prediction in 3D
Jupyter Notebook
29
star
12

LLM4VPR

Can multimodal LLM help visual place recognition?
Python
28
star
13

insta360_ros_driver

A ROS driver for Insta360 cameras, enabling real-time image capture, processing, and publishing in ROS environments.
Python
27
star
14

DeepExplorer

[RSS2023] Metric-Free Exploration for Topological Mapping by Task and Motion Imitation in Feature Space
Python
26
star
15

Self-Supervised-SPARE3D

[CVPR 2022] Self-supervised Spatial Reasoning on Multi-View Line Drawings
Python
24
star
16

RealCity3D

Jupyter Notebook
22
star
17

DeepSoRo

[RA-L/ICRA2020] Real-time Soft Body 3D Proprioception via Deep Vision-based Sensing
Python
22
star
18

FusionSense

Integrates the vision, touch, and common-sense information of foundational models, customized to the agent's perceptual needs.
Python
19
star
19

SNAC

[ICLR2023] Learning Simultaneous Navigation and Construction in Grid Worlds
Python
18
star
20

SeeDo

Human Demo Videos to Robot Action Plans
Python
16
star
21

TF-VPR

Self-supervised place recognition by exploring temporal and feature neighborhoods
Python
15
star
22

DeepParticleRobot

[ICRA'22] A Deep Reinforcement Learning Environment for Particle Robot Navigation and Object Manipulation
Python
13
star
23

pyAprilTag

python wrapper of AprilTag implemented in cv2cg, used for Robot Perception course
C++
10
star
24

NYC-Indoor-VPR

Python
9
star
25

BAGSFit

Primitive Fitting Using Deep Boundary Aware Geometric Segmentation
9
star
26

vis_nav_player

[ROB-GY 6203] Example Visual Navigation Player Code for Course Project
Python
5
star
27

NYC-Event-VPR

Python
5
star
28

Mobile3DPrinting

https://ai4ce.github.io/Mobile3DPrinting/
4
star
29

PointCloudSimulator

Code for simulating 2D point clouds
Python
3
star
30

M3DP-Sim

C++
2
star
31

pyP2Mesh

python wrapper for finding a point's closest point on a triangle mesh
Python
2
star
32

VIP_SelfDrive

Makefile
2
star
33

EgoPAT3Dv2

Python
2
star
34

LUWA

Jupyter Notebook
1
star
35

ai4ce_robot_ROS2_drivers

This repo contains all the ROS2 driver packages modified at AI4CE lab for working with various robots
1
star