• Stars
    star
    273
  • Rank 150,780 (Top 3 %)
  • Language
    C++
  • License
    BSD 3-Clause "New...
  • Created 10 months ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Spatio-Temporal Metric-Semantic SLAM

Khronos

This repository will contain the code for Khronos, our framework for online Spatio-Temporal Metric-Semantic SLAM (SMS).

Khronos

Khronos is a unified approach that can reason about short-term dynamics and long-term changes when performing online metric-semantic simultaneous mapping and localization (SLAM) in dynamic environments. A few instances from Khronos’ spatio-temporal map, representing the scene state at all times, are shown above. Short-term dynamics (left) are shown in magenta and compared against observed human actions over the corresponding time interval. Both humans and inanimate objects (the cart) are detected. Long-term changes (right) are shown for three time instances of the same scene. The earliest instance is at time 0:20 (top right). While the robot is moving through the hallways, a chair is removed and a red cooler is placed on top of the table; these changes are detected as the robot revisits and closes the loop at time 1:52 (bottom right). Lastly, the cooler is removed again, which is detected by the robot at time 3:35.

This project was supported by the Amazon Science Hub “Next-Generation Spatial AI for Human-Centric Robotics” project, the ARL DCIST program, the ONR RAPID program, and the Swiss National Science Foundation (SNSF) grant No. 214489.

Table of Contents

Credits

Note The code will be released here shortly.

Setup

  • Installation
  • Datasets

Examples

  • Running Khronos
  • Visualizing the 4D map

Paper

If you find this useful for your research, please consider citing our paper:

  • Lukas Schmid, Marcus Abate, Yun Chang, and Luca Carlone, "Khronos: A Unified Approach for Spatio-Temporal Metric-Semantic SLAM in Dynamic Environments", in ArXiv Preprint, 2024. [ ArXiv | Video ]
     @misc{Schmid2024Khronos,
        title={Khronos: A Unified Approach for Spatio-Temporal Metric-Semantic SLAM in Dynamic Environments}, 
        author={Lukas Schmid and Marcus Abate and Yun Chang and Luca Carlone},
        year={2024},
        eprint={2402.13817},
        archivePrefix={arXiv},
        primaryClass={cs.RO}
     }

Video

An overview of Khronos is available on YouTube:

Khronos Youtube Video

More Repositories

1

Kimera

Index repo for Kimera code
1,800
star
2

TEASER-plusplus

A fast and robust point cloud registration library
C++
1,753
star
3

Kimera-VIO

Visual Inertial Odometry with SLAM capabilities and 3D Mesh generation.
C++
1,534
star
4

Kimera-Semantics

Real-Time 3D Semantic Reconstruction from 2D data
C++
640
star
5

Hydra

C++
583
star
6

Kimera-RPGO

Robust Pose Graph Optimization
C++
468
star
7

Kimera-VIO-ROS

ROS wrapper for Kimera-VIO
C++
371
star
8

Kimera-Multi

Index repo for Kimera-Multi system
Shell
327
star
9

Loc-NeRF

Monte Carlo Localization using Neural Radiance Fields
Python
259
star
10

Kimera-Multi-Data

A large-scale multi-robot dataset for multi-robot SLAM
149
star
11

GNC-and-ADAPT

Graduated Non-Convexity (GNC) and Adaptive Trimming (ADAPT) algorithms for outlier robust estimation
MATLAB
112
star
12

CertifiablyRobustPerception

Certifiable Outlier-Robust Geometric Perception
MATLAB
107
star
13

PD-MeshNet

Primal-Dual Mesh Convolutional Neural Networks
Python
106
star
14

GlobalOptimizationTutorial

Hands-on Tutorial for Global Optimization in Matlab
MATLAB
104
star
15

llm_scene_understanding

HTML
71
star
16

STRIDE

Solver for Large-Scale Rank-One Semidefinite Relaxations
MATLAB
66
star
17

Clio

66
star
18

VNAV-labs

Labs for MIT 16.485
Python
64
star
19

Spatial-Hash

Minimal C++ library for spatial data structures based on voxel hashing
C++
57
star
20

KISS-Matcher

52
star
21

Hydra-ROS

Hydra ROS Interface
C++
39
star
22

Spark-DSG

Scene Graph API (C++ and Python)
C++
37
star
23

Kimera-VIO-ROS2

C++
36
star
24

Kimera-Multi-LCD

C++
34
star
25

Kimera-VIO-Evaluation

Code to automatically evaluate and tune parameters for Kimera-VIO pipeline.
Python
33
star
26

Kimera-PGMO

C++
33
star
27

ROBIN

C++
31
star
28

config_utilities

Automatic C++ config structs and tools.
C++
28
star
29

pose_graph_tools

C++
26
star
30

pose-baselines

Jupyter Notebook
25
star
31

estimation-contracts

MATLAB
25
star
32

neural_tree

Python
22
star
33

C-3PO

Python
21
star
34

Kimera-Distributed

C++
21
star
35

MiDiffusion

Python
17
star
36

LP2

Long-term Human Trajectory Prediction using 3D DSGs
15
star
37

GlobalOptimization-ICCV2019

ICCV 2019 Tutorial: Global Optimization for Geometric Understanding with Provable Guarantees
TeX
14
star
38

FUSES

C++
11
star
39

Hydra-GNN

Room classification network training and inference code
Python
10
star
40

certifiable_tracking

MATLAB
8
star
41

PACE

Index repo for the PACE project
Python
7
star
42

robotRepresentations-RSS2023

Robot Representations Workshop @ RSS 2023
SCSS
6
star
43

ensemble_pose

Self-training for an ensemble of object pose estimators
Python
5
star
44

ford-paper-params

Parameters for competitor pipelines for the Kimera Multicamera project
5
star
45

CertifiablePerception-RSS2020

Workshop website on Certifiable Robot Perception for RSS2020
TeX
5
star
46

Longterm-Perception-WS

Workshop on Long-term Perception for Autonomy in Dynamic Human-shared Environments
SCSS
4
star
47

ThreedFront

Python
4
star
48

PerceptionMonitoring

Python
3
star
49

dbow2_catkin

CMake
2
star
50

PAL-ICRA2020

Ruby
2
star
51

kimera-multi-vocab

Vocabulary file for Kimera-Multi in the Kimera-Multi dataset
2
star
52

VNAV2020-handouts

Handouts/webpage for VNAV 2020
HTML
1
star