• Stars
    star
    198
  • Rank 196,898 (Top 4 %)
  • Language
    Python
  • Created over 6 years ago
  • Updated about 6 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

3D Human Pose Estimation in RGBD Images for Robotic Task Learning

3D Human Pose Estimation in RGBD Images for Robotic Task Learning

Teaser

This is a ROS node wrapping the approach presented in our paper for estimating 3D Human Pose from a single RGB-D frame. See the project page for additional information.

Requirements

  • Tested with ROS Indigo under Ubuntu 14.04
  • Make sure you have installed the cv_bridge package
  • Our node relies on warped depth maps as input, we recommend to use iai_kinect
  • Get Tensorflow with gpu support for python2.7
  • sudo pip install numpy Pillow scipy numpy opencv-python matplotlib

If you want to use our approach without ROS we refer to forward_pass.py, which applies our approach to the data we provide.

Installation guide

  • Make sure you fulfill the requirements
  • Clone this repository
  • Download the weights (~550 MB) and unzip them into the folder you cloned the repository to. Afterwards there should be a subdirectory ./weights/ containing a *.pkl file and two directories with Tensorflow snapshot models.
  • Run python2.7 forward_pass.py. If it fails there is a problem that is not ROS related (Tensorflow, Other python packages, weights missing, Cuda, GPU, ... )
  • Start the ROS node with python2.7 RosNode.py. For analyzing the predictions we recommend RViz.

Node description

The node subscribes to the following topics (given with its default values):

COLOR_IMAGE_TOPIC = "/kinect2/qhd/image_color"  # undistorted color image of the camera
DEPTH_MAP_TOPIC = "/kinect2/qhd/image_depth_rect"  # depth map warped into the color frame
CAMERA_INFO_TOPIC = '/kinect2/qhd/camera_info'  # ROS topic containing camera calibration K and camera frame

and publishes the detected poses as:

MarkerArray with topic "/human_pose" 
tf.transform with frames "/human_pose/personX/Y"  with X=PERSON_ID Y=KEYPOINT_NAME

Its possible to modify the topic names in the top of RosNode.py. Please not that we only set the location of Tf.transform and leave the orientation unset.

In order to start the node simply run

python2.7 RosNode.py

Citation

If the node helps your research we welcome you to cite our work:

@InProceedings{zw2018,
  author    = {Christian Zimmermann, Tim Welschehold, Christian Dornhege, Wolfram Burgard and Thomas Brox},
  title     = {3D Human Pose Estimation in RGBD Images for Robotic Task Learning},
  booktitle    = "IEEE International Conference on Robotics and Automation (ICRA)",
  year      = {2018},
  url          = "https://lmb.informatik.uni-freiburg.de/projects/rgbd-pose3d/"
}

Maintainers

If you have trouble with the node file an issue or contact:

More Repositories

1

flownet2

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
C++
1,004
star
2

hand3d

Network estimating 3D Handpose from single color images
Python
801
star
3

demon

DeMoN: Depth and Motion Network
Python
574
star
4

freihand

A dataset for estimation of hand pose and shape from single color images.
Python
382
star
5

deeptam

DeepTAM: Deep Tracking and Mapping https://lmb.informatik.uni-freiburg.de/people/zhouh/deeptam/
Python
233
star
6

mv3d

Multi-view 3D Models from Single Images with a Convolutional Network
Python
214
star
7

flownet2-docker

Dockerfile and runscripts for FlowNet 2.0 (estimation of optical flow)
Shell
158
star
8

netdef_models

Repository for different network models related to flow/disparity (ECCV 18)
Python
157
star
9

ogn

Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs
C++
155
star
10

orion

ORION: Orientation-boosted Voxel Nets for 3D Object Recognition
MATLAB
111
star
11

what3d

What Do Single-view 3D Reconstruction Networks Learn?
Python
98
star
12

dispnet-flownet-docker

Dockerfile and runscripts for DispNet and FlowNet1 (estimation of disparity and optical flow)
Shell
87
star
13

Unet-Segmentation

The U-Net Segmentation plugin for Fiji (ImageJ)
Java
87
star
14

robustmvd

Repository for the Robust Multi-View Depth Benchmark
Python
74
star
15

contra-hand

Code in conjunction with the publication 'Contrastive Representation Learning for Hand Shape Estimation'
Python
53
star
16

Multimodal-Future-Prediction

The official repository for the CVPR 2019 paper "Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction"
Python
47
star
17

lmbspecialops

A collection of tensorflow ops
C++
46
star
18

FLN-EPN-RPN

This repository contains the source code of the CVPR 2020 paper: "Multimodal Future Localization and Emergence Prediction for Objects in Egocentric View with a Reachability Prior"
Python
32
star
19

flow_rl

Python
28
star
20

netdef-docker

DispNet3, FlowNet3, FlowNetH, SceneFlowNet -- in Docker
Shell
28
star
21

caffe-unet-docker

The U-Net Segmentation server (caffe_unet) for Docker
Shell
27
star
22

Contrastive-Future-Trajectory-Prediction

The official repository of the ICCV paper "On Exposing the Challenging Long Tail in Future Prediction of Traffic Actors"
Python
25
star
23

locov

Localized Vision-Language Matching for Open-vocabulary Object Detection
Python
19
star
24

unsup-car-dataset

Unsupervised Generation of a Viewpoint Annotated Car Dataset from Videos
MATLAB
19
star
25

FreiPose-docker

FreiPose: A Deep Learning Framework for Precise Animal Motion Capture in 3D Spaces
Dockerfile
18
star
26

optical-flow-2d-data-generation

Caffe(v1)-compatible codebase to generate optical flow training data on-the-fly; used for the IJCV 2018 paper "What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?" (http://dx.doi.org/10.1007/s11263-018-1082-6)
C++
18
star
27

autodispnet

Code for AutoDispNet (ICCV 2019)
Python
17
star
28

cv-exercises

Python
15
star
29

spr-exercises

Jupyter Notebook
12
star
30

td-or-not-td

Code for the paper "TD or not TD: Analyzing the Role of Temporal Differencing in Deep Reinforcement Learning", Artemij Amiranashvili, Alexey Dosovitskiy, Vladlen Koltun and Thomas Brox, ICLR 2018
Python
12
star
31

sf2se3

Repository for SF2SE3: Clustering Scene Flow into SE(3)-Motions via Proposal and Selection
Python
10
star
32

ovqa

Python
10
star
33

understanding_flow_robustness

Official repository for "Towards Understanding Adversarial Robustness of Optical Flow Networks" (CVPR 2022)
Python
9
star
34

neural-point-cloud-diffusion

Official repository for "Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation"
Python
9
star
35

ldce

Official repository for "Latent Diffusion Counterfactual Explanations"
Python
9
star
36

PreFAct

Code and Models for the paper "Learning Representations for Predicting Future Activities"
8
star
37

ROS-packages

A collection of ROS packages for LMB software; DispNet(1+3), FlowNet2, etc.
C++
7
star
38

FreiPose

C++
7
star
39

diffusion-for-ood

Official repository for "Diffusion for Out-of-Distribution Detection on Road Scenes and Beyond". Coming soon.
Python
5
star
40

tfutils

tfutils is a set of tools for training networks with tensorflow
Python
5
star
41

FreiCalib

C++
5
star
42

netdef_slim

A python wrapper for tf to ease creation of network definitions.
Python
4
star
43

iRoCS-Toolbox

n-D Image Analysis libraries and tools
C++
4
star
44

rohl

Python
3
star
45

RecordTool

Python
2
star
46

tree-planting

Official repository for "Climate-sensitive Urban Planning Through Optimization of Tree Placements"
Python
2
star
47

ade-ood

Official repo for the ADE-OoD benchmark.
Python
1
star