Lifting from the Deep
Denis Tome', Chris Russell, Lourdes Agapito
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image, CVPR 2017
This project is licensed under the terms of the GNU GPLv3 license. By using the software, you are agreeing to the terms of the license agreement (link).
Abstract
We propose a unified formulation for the problem of 3D human pose estimation from a single raw RGB image that reasons jointly about 2D joint estimation and 3D pose reconstruction to improve both tasks. We take an integrated approach that fuses probabilistic knowledge of 3D human pose with a multi-stage CNN architecture and uses the knowledge of plausible 3D landmark locations to refine the search for better 2D locations. The entire process is trained end-to-end, is extremely efficient and obtains stateof-the-art results on Human3.6M outperforming previous approaches both on 2D and 3D errors.
Dependencies
The code is compatible with python2.7
Models
For this demo, CPM's caffe-models trained on the MPI datasets (link) are used for 2D pose estimation, whereas for 3D pose estimation our probabilistic 3D pose model is trained on the Human3.6M dataset.
Testing
- First, run
setup.shto retreive the trained models and to install the external utilities. - Run
demo.pyto evaluate the test image.
Additional material
Citation
@InProceedings{Tome_2017_CVPR,
author = {Tome, Denis and Russell, Chris and Agapito, Lourdes},
title = {Lifting From the Deep: Convolutional 3D Pose Estimation From a Single Image},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {July},
year = {2017}
}
Notes
The models provided for the demo are NOT the ones that have been used to generate results for the paper. We are still in the process of converting all the code.
Acknowledgements
This work has been supported by the SecondHands project, funded from the EU Horizon 2020 Research and Innovation programme under grant agreement No 643950.