3D Human Pose Estimation in RGBD Images for Robotic Task Learning
This is a ROS node wrapping the approach presented in our paper for estimating 3D Human Pose from a single RGB-D frame. See the project page for additional information.
Requirements
- Tested with ROS Indigo under Ubuntu 14.04
- Make sure you have installed the cv_bridge package
- Our node relies on warped depth maps as input, we recommend to use iai_kinect
- Get Tensorflow with gpu support for python2.7
- sudo pip install numpy Pillow scipy numpy opencv-python matplotlib
If you want to use our approach without ROS we refer to forward_pass.py
, which applies our approach to the data we provide.
Installation guide
- Make sure you fulfill the requirements
- Clone this repository
- Download the weights (~550 MB) and unzip them into the folder you cloned the repository to. Afterwards there should be a subdirectory
./weights/
containing a*.pkl
file and two directories with Tensorflow snapshot models. - Run
python2.7 forward_pass.py
. If it fails there is a problem that is not ROS related (Tensorflow, Other python packages, weights missing, Cuda, GPU, ... ) - Start the ROS node with
python2.7 RosNode.py
. For analyzing the predictions we recommend RViz.
Node description
The node subscribes to the following topics (given with its default values):
COLOR_IMAGE_TOPIC = "/kinect2/qhd/image_color" # undistorted color image of the camera
DEPTH_MAP_TOPIC = "/kinect2/qhd/image_depth_rect" # depth map warped into the color frame
CAMERA_INFO_TOPIC = '/kinect2/qhd/camera_info' # ROS topic containing camera calibration K and camera frame
and publishes the detected poses as:
MarkerArray with topic "/human_pose"
tf.transform with frames "/human_pose/personX/Y" with X=PERSON_ID Y=KEYPOINT_NAME
Its possible to modify the topic names in the top of RosNode.py
.
Please not that we only set the location of Tf.transform and leave the orientation unset.
In order to start the node simply run
python2.7 RosNode.py
Citation
If the node helps your research we welcome you to cite our work:
@InProceedings{zw2018,
author = {Christian Zimmermann, Tim Welschehold, Christian Dornhege, Wolfram Burgard and Thomas Brox},
title = {3D Human Pose Estimation in RGBD Images for Robotic Task Learning},
booktitle = "IEEE International Conference on Robotics and Automation (ICRA)",
year = {2018},
url = "https://lmb.informatik.uni-freiburg.de/projects/rgbd-pose3d/"
}
Maintainers
If you have trouble with the node file an issue or contact:
- Tim Welschehold, Autonomous Intelligent Systems, University of Freiburg
- Christian Zimmermann, Pattern Recognition and Image Processing, University of Freiburg