• Stars
    star
    158
  • Rank 237,131 (Top 5 %)
  • Language
    C++
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated 7 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

ROS 2 wrapper for the ZED SDK

Stereolabs
ROS2 wrapper

ROS2 packages for using Stereolabs ZED Camera cameras.
ROS2 Foxy Fitzroy (Ubuntu 20.04) - ROS2 Humble Hawksbill (Ubuntu 22.04)


This package lets you use the ZED stereo cameras with ROS2. It provides access to the following data:

  • Left and right rectified/unrectified images
  • Depth data
  • Colored 3D point cloud
  • Position and Mapping (with GNSS data fusion)
  • Sensors data (not available with ZED)
  • Detected objects (not available with ZED)
  • Persons skeleton (not available with ZED)

More information

Installation

Prerequisites

Build the package

The zed_ros2_wrapper is a colcon package.

Note: If you havenโ€™t set up your colcon workspace yet, please follow this short tutorial.

To install the zed_ros2_wrapper, open a bash terminal, clone the package from Github, and build it:

mkdir -p ~/ros2_ws/src/ # create your workspace if it does not exist
cd ~/ros2_ws/src/ #use your current ros2 workspace folder
git clone  --recursive https://github.com/stereolabs/zed-ros2-wrapper.git
cd ..
sudo apt update
rosdep install --from-paths src --ignore-src -r -y # install dependencies
colcon build --symlink-install --cmake-args=-DCMAKE_BUILD_TYPE=Release --parallel-workers $(nproc) # build the workspace
echo source $(pwd)/install/local_setup.bash >> ~/.bashrc # automatically source the installation in every new bash (optional)
source ~/.bashrc

Note: If rosdep is missing you can install it with:

sudo apt-get install python3-rosdep python3-rosinstall-generator python3-vcstool python3-rosinstall build-essential

Note: The option --symlink-install is very important, it allows to use symlinks instead of copying files to the ROS2 folders during the installation, where possible. Each package in ROS2 must be installed and all the files used by the nodes must be copied into the installation folders. Using symlinks allows you to modify them in your workspace, reflecting the modification during the next executions without the needing to issue a new colcon build command. This is true only for all the files that don't need to be compiled (Python scripts, configurations, etc.).

Note: If you are using a different console interface like zsh, you have to change the source command as follows: echo source $(pwd)/install/local_setup.zsh >> ~/.zshrc and source ~/.zshrc.

Starting the ZED node

To start the ZED node, open a bash terminal and use the CLI command ros2 launch:

$ ros2 launch zed_wrapper zed_camera.launch.py camera_model:=<camera_model>

Replace <camera_model> with the model of the camera that you are using: 'zed', 'zedm', 'zed2', 'zed2i', 'zedx', 'zedxm'.

The zed_camera.launch.py is a Python launch scripts that automatically start the ZED node using "manual composition". The parameters for the indicated camera model are loaded from the relative "YAML files". A Robot State Publisher node is started to publish the camera static links and joints loaded from the URDF model associated with the camera model.

Note: You can set your own configurations modifying the parameters in the files common.yaml, zed.yaml zedm.yaml, zed2.yaml, zed2i.yaml, zedx.yaml, and zedxm.yaml available in the folder zed_wrapper/config.

For full descriptions of each parameter, follow the complete guide here.

Rviz visualization

To start a pre-configured Rviz environment and visualize the data of all ZED cameras, we provide in the zed-ros2-examples repository. You'll see there more advanced examples and visualisation that demonstrate depth, point clouds, odometry, object detection, etc.

You can also quickly check that your depth data is correctly retrieved in rviz with rviz2 -d ./zed_wrapper/config/rviz2/<your camera model>.rviz. Be aware that rviz subscribes to numerous ROS topics, which can potentially impact the performance of your application compared to when it runs without rviz.

More features

SVO recording

SVO recording can be started and stopped while the ZED node is running using the service start_svo_recording and the service stop_svo_recording. More information

Object Detection

The Object Detection can be enabled automatically when the node start by setting the parameter object_detection/od_enabled to true in the file common.yaml. The Object Detection can be enabled/disabled manually by calling the services enable_obj_det.

Body Tracking

The Body Tracking can be enabled automatically when the node start by setting the parameter body_tracking/bt_enabled to true in the file common.yaml.

The Object Detection module is not available on the very first generation of ZED cameras.

Spatial Mapping

The Spatial Mapping can be enabled automatically when the node start setting the parameter mapping/mapping_enabled to true in the file common.yaml. The Spatial Mapping can be enabled/disabled manually calling the services enable_mapping.

GNSS fusion

The ZED ROS2 Wrapper can subscribe to a NavSatFix topic and fuse GNSS data information with Positional Tracking information to obtain a precise robot localization referred to Earth coordinates. To enable GNSS fusion set the parameter gnss_fusion.gnss_fusion_enabled to true. It is important that you set the correct gnss_frame parameter when launching the node, e.g. gnss_frame:='gnss_link'. The services toLL and fromLL can be used to convert Latitude/Longitude coordinates to robot map coordinates.

2D mode

For robots moving on a planar surface it is possible to activate the "2D mode" (parameter pos_tracking/two_d_mode in common.yaml). The value of the coordinate Z for odometry and pose will have a fixed value (parameter pos_tracking/fixed_z_value in common.yaml). Roll and pitch and relative velocities will be fixed to zero.

Examples and Tutorials

Examples and tutorials are provided to better understand how to use the ZED wrapper and how to integrate it in the ROS2 framework. See the zed-ros2-examples repository

Rviz2 visualization examples

  • Example launch files to start a preconfigured instance of Rviz displaying all the ZED Wrapper node information: zed_display_rviz2
  • ROS2 plugin for ZED2 to visualize the results of the Object Detection and Body Tracking modules (bounding boxes and skeletons): rviz-plugin-zed-od

Tutorials

Update the local repository

To update the repository to the latest release, use the following command that will retrieve the latest commits of zed-ros2-wrapper and of all the submodules:

git checkout master # if you are not on the main branch  
git pull --recurse-submodules # update recursively all the submodules

Clean the cache of your colcon workspace before compiling with the colcon build command to be sure that everything will work as expected:

cd <ros2_workspace_root> # replace with your workspace folder, for example ~/ros2_ws/src/
rm -r install
rm -r build
rm -r log
colcon build --symlink-install --cmake-args=-DCMAKE_BUILD_TYPE=Release --parallel-workers $(nproc)

Known issues

[ROS2 Foxy] Image Transport and topic subscriptions

There is an IMPORTANT issue in ROS2 Foxy with the function CameraPublisher::getNumSubscribers preventing the correct counting of the number of nodes subscribing one of the topics published by an image_transport::CameraPublisher object and hence stopping the correct publishing of the subscribed topics.

The only known solution is to install the exact version v3.0.0 of the image_transport package, published on 2021-05-26, that contains the fix for this issue.

To install the working version from the sources:

cd <colcon_workspace>/src # Access the source folder of your colcon workspace
git clone https://github.com/ros-perception/image_common.git --branch 3.0.0 --single-branch # clone the "v3.0.0" branch of the "image_common" repository
cd <colcon_workspace> # Go back to the root of your colcon workspace
colcon build --symlink-install # Compile everything and install

Close the console and re-open it to apply the modifications.

[ROS2 Foxy] Image Transport Plugins and compressed topics

The image_transport_plugins package is not correctly working with ROS2 Foxy (see here, here, here, and here). We suggest you remove it to avoid many annoying warning messages until the ROS2 developers do not fix it or we find a workaround:

sudo apt remove ros-foxy-image-transport-plugins ros-foxy-compressed-depth-image-transport ros-foxy-compressed-image-transport

More Repositories

1

zed-sdk

โšก๏ธThe spatial perception framework for rapidly building smart robots and spaces
C++
784
star
2

zed-ros-wrapper

ROS wrapper for the ZED SDK
C++
440
star
3

zed-unity

ZED SDK Unity plugin
C#
252
star
4

zed-python-api

Python API for the ZED SDK
Cython
206
star
5

zed-yolo

3D Object detection using Yolo and the ZED in Python and C++
Python
159
star
6

zed-opencv

ZED SDK interface sample for OpenCV
C++
137
star
7

zed-open-capture

Low level Linux camera driver for the ZED USB3 stereo cameras. API docs available here:
C++
101
star
8

zed-ros-examples

Examples for the ZED SDK ROS wrapper
C++
85
star
9

zed-unreal-examples

Stereolabs ZED - UE4 Examples
C++
77
star
10

zed-tensorflow

3D Object detection using the ZED and Tensorflow
Python
77
star
11

zed-UE5

ZED plugin and examples for Unreal Engine 5 (Standard Engine)
C++
65
star
12

zed-gstreamer

Package of GStreamer elements to interface with ZED Cameras
C++
62
star
13

zed-pcl

ZED SDK interface sample for PCL, the Point Cloud Library
C++
58
star
14

zed-multi-camera

ZED SDK sample with multiple ZEDs
C++
52
star
15

zed-docker

Docker images for the ZED SDK
Dockerfile
51
star
16

zed-pytorch

3D Object detection using the ZED and Pytorch
Jupyter Notebook
48
star
17

zed-openpose

Real-time 3D multi-person with OpenPose and the ZED
C++
42
star
18

zed-unreal-plugin

Stereolabs ZED - UE4 Plugin
C++
41
star
19

zed-opencv-native

Native stereo capture with OpenCV
C++
38
star
20

zed-ros2-examples

A collection of examples and tutorials to illustrate how to better use the ZED cameras in the ROS2 framework
C++
33
star
21

zed-livelink

ZED LiveLink Plugin for Unreal
C++
32
star
22

zed-matlab

ZED SDK interface sample for Matlab
C++
28
star
23

zed-oculus

ZED Viewer for Oculus Rift
C++
27
star
24

zed-aruco

ZED SDK samples using ArUco tag detection
C++
21
star
25

zed-tracking-viewer

[DEPRECATED] ZED SDK sample to show the tracking capability
C++
17
star
26

zed-csharp-api

C# API for the ZED SDK
C#
10
star
27

zed-3d-social-distancing

Using 3D Cameras to Monitor Social Distancing
C++
10
star
28

zed-ros-interfaces

ROS interfaces for the ZED ROS Wrapper
CMake
9
star
29

zed-ros2-interfaces

ROS2 interfaces for the ZED ROS2 Wrapper
CMake
5
star
30

zed-hub-examples

ZEDHub app tutorials
Python
5
star
31

zed-isaac-sim

NVIDIA Isaac Sim integration for ZED SDK
Python
5
star
32

zed-unity-livelink

ZED Livelink plugin for Unity
C++
5
star
33

zed-opengl

ZED SDK interface sample for OpenGL
C++
2
star
34

zed-openni

C
2
star
35

zedx-one-capture

Camera control API for ZED X One GMSL2 cameras from Stereolabs
C++
2
star