• Stars
    star
    128
  • Rank 272,101 (Top 6 %)
  • Language
    C++
  • License
    BSD 3-Clause "New...
  • Created over 7 years ago
  • Updated over 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A collection of segmentation methods working on depth images

depth_segmentation

This package provides geometric segmentation of depth images and an interface to semantic instance segmentation, where the output of a semantic instance segmentation of RGB images gets combined with the geometric instance segmentation. For the later case we assign each geometric segment a semantic label as well. TODO Add image(s)

If you are interested in a global segmentation map, please also take a look at voxblox-plusplus.

Installation

In your terminal, define the installed ROS version and name of the catkin workspace to use:

export ROS_VERSION=kinetic # (Ubuntu 16.04: kinetic, Ubuntu 18.04: melodic)
export CATKIN_WS=~/catkin_ws

If you don't have a catkin workspace yet, create a new one:

mkdir -p $CATKIN_WS/src && cd $CATKIN_WS
catkin init
catkin config --extend /opt/ros/$ROS_VERSION --merge-devel 
catkin config --cmake-args -DCMAKE_CXX_STANDARD=14 -DCMAKE_BUILD_TYPE=Release
wstool init src

Note: If you already have a catkin workspace, ensure that its devel space layout is merged. If you do catkin config in your workspace the output should include:

Devel Space Layout:          merged

Clone the depth_segmentation repository over HTTPS (no Github account required):

cd $CATKIN_WS/src
git clone --recurse-submodules https://github.com/ethz-asl/depth_segmentation.git

Alternatively, clone over SSH (Github account required):

cd $CATKIN_WS/src
git clone --recurse-submodules [email protected]:ethz-asl/depth_segmentation.git

Automatically fetch dependencies:

wstool merge -t . depth_segmentation/dependencies.rosinstall
wstool update

Build and source the packages:

catkin build depth_segmentation
source ../devel/setup.bash # (bash shell: ../devel/setup.bash,  zsh shell: ../devel/setup.zsh)

To compile it with Mask R-CNN support you'll need to set the WITH_MASKRCNNROS to ON in the CMakeLists.txt file:

set(WITH_MASKRCNNROS ON)

Usage

The two use cases can be started as described below.

Geometric Segmentation

If you only want geometric segmentation, use:

roslaunch depth_segmentation depth_segmenatation.launch

You'll need to adjust the ros topic names in the sensor_topics_file (by default this is in depth_segmentation//cfg/primesense_topics.yaml). The depth segmentation parameters can be adjusted via dynamic reconfigure or in the depth_segmentation_params_file directly.

Combined Geometric Segmentation with Semantics

To additionally run the semantic segmentation you can use this command:

roslaunch depth_segmentation semantic_instance_segmentation.launch

NOTE This only works if you have compiled depth_segmentation with Mask R-CNN enabled (WITH_MASKRCNNROS=ON).

Citing

If you use this, please cite:

  • Fadri Furrer, Tonci Novkovic, Marius Fehr, Abel Gawel, Margarita Grinvald, Torsten Sattler, Roland Siegwart, Juan Nieto, Incremental Object Database: Building 3D Models from Multiple Partial Observations, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018. [PDF] [Video]
@INPROCEEDINGS{8594391, 
author={F. {Furrer} and T. {Novkovic} and M. {Fehr} and A. {Gawel} and M. {Grinvald} and T. {Sattler} and R. {Siegwart} and J. {Nieto}}, 
booktitle={2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, 
title={Incremental Object Database: Building 3D Models from Multiple Partial Observations}, 
year={2018},
pages={6835-6842}, 
keywords={feature extraction;image colour analysis;image reconstruction;image representation;image segmentation;mobile agents;object detection;solid modelling;multiple partial observations;incremental object database;indoor scenes;merged models;object model;observed instances;segmented RGB-D images;global segmentation map;3D models;mobile agent;Image segmentation;Databases;Three-dimensional displays;GSM;Shape;Image reconstruction;Solid modeling}, 
doi={10.1109/IROS.2018.8594391}, 
ISSN={2153-0866}, 
month={Oct},}

If you also use the semantic segmentation, additionally cite:

  • Margarita Grinvald, Fadri Furrer, Tonci Novkovic, Jen Jen Chung, Cesar Cadena, Roland Siegwart, Juan Nieto, Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery, IEEE Robotics and Automation Letters, 2019. [PDF] [Video]
@article{grinvald2019volumetric,
  title={{Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery}},
  author={Grinvald, Margarita and Furrer, Fadri and Novkovic, Tonci and Chung, Jen Jen and Cadena, Cesar and Siegwart, Roland and Nieto, Juan},
  journal={IEEE Robotics and Automation Letters},
  year={2019},
  note={Accepted}
}

License

The code is available under the BSD-3-Clause license.

More Repositories

1

kalibr

The Kalibr visual-inertial calibration toolbox
C++
4,067
star
2

maplab

A Modular and Multi-Modal Mapping Framework
C++
2,471
star
3

voxblox

A library for flexible voxel-based mapping, mainly focusing on truncated and Euclidean signed distance fields.
C++
1,260
star
4

rotors_simulator

RotorS is a UAV gazebo simulator
C++
1,168
star
5

okvis

OKVIS: Open Keyframe-based Visual-Inertial SLAM.
C++
1,158
star
6

rovio

C++
1,098
star
7

segmap

A map representation based on 3D segments
C++
1,045
star
8

ethzasl_msf

MSF - Modular framework for multi sensor fusion based on an Extended Kalman Filter (EKF)
C++
961
star
9

lidar_align

A simple method for finding the extrinsic calibration between a 3D lidar and a 6-dof pose sensor
C++
773
star
10

hfnet

From Coarse to Fine: Robust Hierarchical Localization at Large Scale with HF-Net (https://arxiv.org/abs/1812.03506)
Python
739
star
11

aerial_mapper

Real-time Dense Point Cloud, Digital Surface Map (DSM) and (Ortho-)Mosaic Generation for UAVs
C++
524
star
12

robust_point_cloud_registration

Robust Point Cloud Registration Using Iterative Probabilistic Data Associations ("Robust ICP")
C++
513
star
13

voxgraph

Voxblox-based Pose graph optimization
C++
509
star
14

mav_trajectory_generation

Polynomial trajectory generation and optimization, especially for rotary-wing MAVs.
C++
508
star
15

mav_active_3d_planning

Modular framework for online informative path planning.
C++
507
star
16

polygon_coverage_planning

Coverage planning in general polygons with holes.
C++
458
star
17

mav_voxblox_planning

MAV planning tools using voxblox as the map representation.
Makefile
440
star
18

hand_eye_calibration

Python tools to perform time-synchronization and hand-eye calibration.
Python
422
star
19

voxblox-plusplus

A volumetric object-level semantic mapping framework.
C++
390
star
20

dynablox

Real-time detection of diverse dynamic objects in complex environments.
C++
368
star
21

mav_control_rw

Control strategies for rotary wing Micro Aerial Vehicles using ROS
C
324
star
22

ethzasl_sensor_fusion

time delay single and multi sensor fusion framework based on an EKF
C++
322
star
23

nbvplanner

A real-time capable exploration and inspection path planner (next best view planning)
C++
295
star
24

ethzasl_icp_mapping

3D mapping tools for robotic applications
C++
268
star
25

panoptic_mapping

A flexible submap-based framework towards spatio-temporally consistent volumetric mapping and scene understanding.
C++
266
star
26

okvis_ros

OKVIS: Open Keyframe-based Visual-Inertial SLAM (ROS Version)
C++
256
star
27

laser_slam

This package provides an end-to-end system to laser-based graph SLAM using laser point clouds.
C++
246
star
28

versavis

An Open Versatile Multi-Camera Visual-Inertial Sensor Suite
C++
240
star
29

image_undistort

A compact package for undistorting images directly from kalibr calibration files. Can also perform dense stereo estimation
C++
239
star
30

kitti_to_rosbag

Dataset tools for working with the KITTI dataset raw data ( http://www.cvlibs.net/datasets/kitti/raw_data.php ) and converting it to a ROS bag. Also allows a library for direct access to poses, velodyne scans, and images.
C++
239
star
31

ethzasl_ptam

Modified version of Parallel Tracking and Mapping (PTAM)
C++
235
star
32

geodetic_utils

Simple library for converting coordinates to/from several geodetic frames (lat/lon, ECEF, ENU, NED, etc.)
C++
233
star
33

wavemap

Fast, efficient and accurate multi-resolution, multi-sensor 3D occupancy mapping
C++
226
star
34

cblox

Voxblox-based submapping
C++
202
star
35

aslam_cv2

C++
196
star
36

volumetric_mapping

A repository for 3D volumetric (occupancy) maps, providing a generic interface for disparity map and pointcloud insertion, and support for custom sensor error models.
C++
183
star
37

vgn

Real-time 6 DOF grasp detection in clutter.
Python
181
star
38

hierarchical_loc

Deep image retrieval for efficient 6-DoF localization
Python
172
star
39

orb_slam_2_ros

ROS interface for ORBSLAM2!!
C++
171
star
40

glocal_exploration

Efficient local and global exploration on submap collections with changing past pose estimates.
C++
165
star
41

mav_dji_ros_interface

Interface of DJI autopilot based on its OSDK (3.2)
C++
156
star
42

programming_guidelines

This repository contains style-guides, discussions, eclipse/emacs auto-formatter for commonly used programming languages
Emacs Lisp
139
star
43

lidar_undistortion

Catkin package that provides lidar motion undistortion based on an external 6DoF pose estimation input.
C++
138
star
44

tsdf-plusplus

TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction
C++
135
star
45

odom_predictor

Integrates an IMU to predict future odometry readings
C++
134
star
46

neuralblox

Real-time Neural Representation Fusion for Robust Volumetric Mapping
Python
126
star
47

grid_map_geo

Geolocalization for grid map using GDAL.
C++
111
star
48

phaser

A robust pointcloud registration pipeline based on correlation.
C++
106
star
49

eth_supermegabot

Instructions for ETH center for robotics summer school 2019.
Python
103
star
50

StructuralInspectionPlanner

ASL Structural Inspection Planner
C++
103
star
51

waypoint_navigator

Stand-alone waypoint navigator
C++
96
star
52

ethzasl_xsens_driver

Driver for xsens IMUs
Python
96
star
53

mav_tools_public

General launch files, parameters and wiki entries on our systems and related issues
95
star
54

reinmav-gym

Reinforcement Learning framework for MAVs using the OpenAI Gym environment
Python
90
star
55

minkindr

A minimal library for transformations, following the kindr interface. Uses active quaternions of rotation in Hamilton notation.
C++
88
star
56

cuckoo_time_translator

algorithms for synchronizing clocks
C++
87
star
57

data-driven-dynamics

Data Driven Dynamics Modeling for Aerial Vehicles
Python
86
star
58

sl_sensor

SL Sensor: An open-source, real-time and ROS-based structured light sensor for high accuracy construction robotic applications
C++
84
star
59

ethz_piksi_ros

ROS drivers for the Piksi RTK GPS module
C++
81
star
60

unreal_airsim

Simulation interface to Unreal Engine 4 based on the AirSim plugin.
C++
80
star
61

vicon_bridge

This is a driver providing data from VICON motion capture systems. It is based on the vicon_mocap package from the starmac stacks. Additionally, it can handle multiple subjects / segments and allows to calibrate an origin of the vehicle(s) as this is somehow tedious with the VICON Tracker.
C++
80
star
62

ros-system-monitor

System monitoring tools for ROS.
Python
80
star
63

voxblox_ground_truth

Create ground truth voxblox maps from Gazebo worlds or .ply files
C++
76
star
64

navrep

Python
73
star
65

curves

A library of curves for estimation.
C++
72
star
66

schweizer_messer

Programming tools for robotics.
C++
65
star
67

time_autosync

Automatically syncs a camera to a rigidly attached IMUs time frame
C++
63
star
68

unreal_cv_ros

Unreal CV ROS Perception Simulator
Python
62
star
69

ai_for_robotics

Programming Exercises Accompanying the Lecture "Artificial Intelligence for Robotics"
Python
60
star
70

lcd

Line Clustering and Description for Place Recognition
C++
59
star
71

trajectory_toolkit

Python tool for analyzing and evaluating trajectory data
Python
59
star
72

dataset_tools

Loader for the generic ASL dataset formats.
MATLAB
58
star
73

rl-navigation

OpenEdge ABL
57
star
74

asl-student-templates

Templates and overview information for student projects at ASL
PostScript
56
star
75

libseekthermal

Driver library for Seek Thermal imaging devices
C++
55
star
76

plotty

matplotlib-cpp with Eigen interfaces.
C++
51
star
77

3d_vsg

3D Variable Scene Graphs for long-term semantic scene change prediction.
Python
49
star
78

terrain-navigation

Repository for Safe Low Altitude Navigation in steep terrain for fixed-wing Aerial Vehicles (RA-L Submission)
C++
49
star
79

forest_gen

Generates randomized Poisson forests to use for UAV collision avoidance evaluations.
Python
47
star
80

sampling_based_control

Jupyter Notebook
47
star
81

tmplanner

Terrain monitoring planner
C++
45
star
82

reactive_avoidance

Reactive obstacle avoidance using raytracing or lidars
C++
45
star
83

3d3l

Deep Learned Keypoint Detection and Description for 3D LiDARs
Python
44
star
84

visensor_node

Visual inertial SLAM sensor ROS node.
C++
43
star
85

mav_comm

This repository contains message and service definitions used for mavs. All future message definitions go in here, existing ones in other stacks should be moved here where possible.
C++
43
star
86

maplab_rovio

Hard-fork of ROVIO to integrate localization.
C++
41
star
87

autolabel

A project for computing high-quality ground truth training examples for RGB-D data.
Python
41
star
88

active_grasp

Closed-loop next-best view planning for grasp detection in clutter.
Python
41
star
89

two_state_information_filter

C++
41
star
90

ssc_exploration

Incremental 3D Scene Completion for Safe and Efficient Exploration Mapping and Planning
41
star
91

Learn-to-Calibrate

We utilize deep reinforcement learning to obtain favorable trajectories for visual-inertial system calibration.
C++
41
star
92

rtklibros

rtklib with ros interfacing and adapted feedback from external Kalman filter
C
40
star
93

cvae_exploration_planning

Learning informed sampling distributions and information gains for efficient exploration planning.
Python
40
star
94

libvisensor

Low level hardware driver for the visual inertial SLAM sensor.
C++
39
star
95

mav_gtsam_estimator

A GTSAM based state estimation framework.
C++
39
star
96

mav_system_identification

Matlab scripts to perform system identification for muti-rotor systems
MATLAB
38
star
97

x-view

38
star
98

matlab_trajectory_tools

Tools for enabling quick display and analysis of trajectories and transformations in Matlab.
MATLAB
38
star
99

fgsp

Jupyter Notebook
38
star
100

3dsnet

3DSNet: Unsupervised Shape-to-shape 3D Style Transfer
C++
37
star