State-of-the-art methods on monocular 3D pose estimation / 3D mesh recovery

Benchmark Leaderboard and state-of-the-art method list on monocular 3D pose estimation / 3D mesh recovery

Feel free to contribute!

Contents

• Benchmark Leaderboard
• arXiv Papers
• Conference Papers
  • 2020: CVPR, ECCV, Others
  • 2019: CVPR, ICCV
  • 2018: CVPR, ECCV, Others
  • 2017: CVPR, ICCV
  • 2016: CVPR, ECCV
• Journal Papers
• Datasets
• Other Related Papers

Benchmark Leaderboard

Evaluation Matrix

See evaluation_matrix.py in Numpy / Pytorch to get more details on evaluation matrix for 3D pose estimation / mesh recovery. In general, MPJPE and PA-MPJPE are mean 3D joint errors. The lower, the better.

Comparisons on Human3.6M.

See datasets/Human3.6M to get more details on Human3.6M

State-of-the-art 3D pose estimation methods

State-of-the-art 3D mesh recovery methods

See datasets/3DPW to get more details on 3DPW

Evaluation Datasets of MPJPE: Human3.6M | Human3.6M | HumanEva-I | 3DPW

Valuable Code

Differential Renderer

NMR CVPR18

Neural 3D Mesh Renderer

Hiroharu Kato, Yoshitaka Ushiku, Tatsuya Harada [conf] [Project] [Tensorflow - official] [Pytorch]

Method Zoo

arXiv Papers

[2008.12272] CenterHMR: a Bottom-up Single-shot Method for Multi-person 3D Mesh Recovery from a Single Image

Yu Sun, Qian Bao, Wu Liu, Yili Fu, Tao Mei

[2004.13985] Motion Guided 3D Pose Estimation from Videos

Jingbo Wang, Sijie Yan, Yuanjun Xiong, Dahua Lin

[2003.10350] Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows

Andrei Zanfir, Eduard Gabriel Bazavan, Hongyi Xu, Bill Freeman, Rahul Sukthankar, Cristian Sminchisescu

[2003.10873] EllipBody: A Light-weight and Part-based Representation for Human Pose and Shape Recovery

Min Wang, Feng Qiu, Wentao Liu, Chen Qian, Xiaowei Zhou, Lizhuang Ma

[2004.01166] Bodies at Rest: 3D Human Pose and Shape Estimation from a Pressure Image using Synthetic Data

Henry M. Clever, Zackory Erickson, Ariel Kapusta, Greg Turk, C. Karen Liu, Charles C. Kemp

[2004.02186] Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation

Edoardo Remelli, Shangchen Han, Sina Honari, Pascal Fua, Robert Wang

[2004.03143] Predicting Camera Viewpoint Improves Cross-dataset Generalization for 3D Human Pose Estimation

Zhe Wang, Daeyun Shin, Charless C. Fowlkes

[2004.03989] Multi-Person Absolute 3D Human Pose Estimation with Weak Depth Supervision

Marton Veges, Andras Lorincz
[Code]

[2004.05275] Multi-View Matching (MVM): Facilitating Multi-Person 3D Pose Estimation Learning with Action-Frozen People Video

Yeji Shen, C.-C. Jay Kuo

[2004.05815] MulayCap: Multi-layer Human Performance Capture Using A Monocular Video Camera

Zhaoqi Su, Weilin Wan, Tao Yu, Lingjie Liu, Lu Fang, Wenping Wang, Yebin Liu

[2004.03686] Exemplar Fine-Tuning for 3D Human Pose Fitting Towards In-the-Wild 3D Human Pose Estimation

Hanbyul Joo, Natalia Neverova, Andrea Vedaldi

Conference Papers

2021 ICCV

ROMP

Monocular, One-stage, Regression of Multiple 3D People
Yu Sun, Qian Bao, Wu Liu, Yili Fu, Michael J. Black, Tao Mei
[arxiv] [Pytorch code]

2020 ECCV

Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose

Hongsuk Choi, Gyeongsik Moon, and Kyoung Mu Lee
[arxiv]

Full-Body Awareness from Partial Observations

Chris Rockwell and David F. Fouhey
[paper]

Unsupervised Human 3D Pose Representation with Viewpoint and Pose Disentanglement

Qiang Nie, Ziwei Liu, Yunhui Liu
[arxiv]

2020 CVPR

ETD

Cascaded Deep Monocular 3D Human Pose Estimation With Evolutionary Training Data
Shichao Li, Lei Ke, Kevin Pratama, Yu-Wing Tai, Chi-Keung Tang, Kwang-Ting Cheng
[conf] [Pytorch code]

DKA

Deep Kinematics Analysis for Monocular 3D Human Pose Estimation
Jingwei Xu, Zhenbo Yu, Bingbing Ni, Jiancheng Yang, Xiaokang Yang, Wenjun Zhang
[conf]

3D Human Mesh Regression With Dense Correspondence

Wang Zeng, Wanli Ouyang, Ping Luo, Wentao Liu, Xiaogang Wang
[conf]

Object-Occluded Human Shape and Pose Estimation from a Single Color Image

Tianshu Zhang, Buzhen Huang and Yangang Wang
[pdf]

Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild

Umar Iqbal, Pavlo Molchanov, Jan Kautz
[arxiv]

Attention Mechanism Exploits Temporal Contexts: Real-time 3D Human Pose Reconstruction

Ruixu Liu1, Ju Shen1, He Wang1, Chen Chen2, Sen-ching Cheung3, Vijayan Asari1
[pdf]

Multiview-Consistent Semi-Supervised Learning for 3D Human Pose Estimation

Rahul Mitra, Nitesh B. Gundavarapu, Abhishek Sharma, Arjun Jain
[conf]

VIBE

VIBE: Video Inference for Human Body Pose and Shape Estimation
Muhammed Kocabas, Nikos Athanasiou, Michael J. Black
[arxiv] [Pytorch code]

Multiperson

Coherent Reconstruction of Multiple Humans From a Single Image*
Wen Jiang, Nikos Kolotouros, Georgios Pavlakos, Xiaowei Zhou, Kostas Daniilidis
[conf]

ActiveMoCap

ActiveMoCap: Optimized Viewpoint Selection for Active Human Motion Capture
Sena Kiciroglu, Helge Rhodin, Sudipta N. Sinha, Mathieu Salzmann, Pascal Fua
[conf]

PandaNet

PandaNet: Anchor-Based Single-Shot Multi-Person 3D Pose Estimation
Abdallah Benzine, Florian Chabot, Bertrand Luvison, Quoc Cuong Pham, Catherine Achard
[conf]

LoCO

Compressed Volumetric Heatmaps for Multi-Person 3D Pose Estimation
Matteo Fabbri, Fabio Lanzi, Simone Calderara, Stefano Alletto, Rita Cucchiara
[arxiv] [Code]

Pgp

Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis
Jogendra Nath Kundu, Siddharth Seth, Varun Jampani, Mugalodi Rakesh, R. Venkatesh Babu, Anirban Chakraborty
[arxiv] [Project]

2020-Others

AAAI2020 - 3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training

Yu Cheng, Bo Yang, Bo Wang, Robby T. Tan [arxiv]

2019 ICCV

DSD-SATN

Human Mesh Recovery From Monocular Images via a Skeleton-Disentangled Representation

Yu Sun, Yun Ye, Wu Liu, Wenpeng Gao, Yili Fu, Tao Mei

[conf] [Pytorch code]

SPIN

Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop

Nikos Kolotouros, Georgios Pavlakos, Michael J. Black, Kostas Daniilidis

[conf] [Pytorch code]

TexturePose

TexturePose: Supervising Human Mesh Estimation With Texture Consistency

Georgios Pavlakos, Nikos Kolotouros, Kostas Daniilidis

[conf] [Pytorch code]

DenseRaC

DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare

Yuanlu Xu, Song-Chun Zhu, Tony Tung

[conf]

BSFMV

On Boosting Single-Frame 3D Human Pose Estimation via Monocular Videos

Zhi Li, Xuan Wang, Fei Wang, Peilin Jiang [conf]

STR-GCN

Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks

Yujun Cai, Liuhao Ge, Jun Liu, Jianfei Cai, Tat-Jen Cham, Junsong Yuan, Nadia Magnenat Thalmann [conf]

GOR

Monocular 3D Human Pose Estimation by Generation and Ordinal Ranking

Saurabh Sharma, Pavan Teja Varigonda, Prashast Bindal, Abhishek Sharma, Arjun Jain [conf]

C3DPO

C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion

David Novotny, Nikhila Ravi, Benjamin Graham, Natalia Neverova, Andrea Vedaldi [conf]

HEMlets

HEMlets Pose: Learning Part-Centric Heatmap Triplets for Accurate 3D Human Pose Estimation

Kun Zhou, Xiaoguang Han, Nianjuan Jiang, Kui Jia, Jiangbo Lu [conf]

ONS

Optimizing Network Structure for 3D Human Pose Estimation

Hai Ci, Chunyu Wang, Xiaoxuan Ma, Yizhou Wang [conf]

DK

Distill Knowledge From NRSfM for Weakly Supervised 3D Pose Learning

Chaoyang Wang, Chen Kong, Simon Lucey [conf]

OANet

Occlusion-Aware Networks for 3D Human Pose Estimation in Video

Yu Cheng, Bo Yang, Bo Wang, Wending Yan, Robby T. Tan

[conf]

3DMPPE

Camera Distance-Aware Top-Down Approach for 3D Multi-Person Pose Estimation From a Single RGB Image

Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee

[conf] [Pytorch code]

2019 CVPR

GCMR

Convolutional Mesh Regression for Single-Image Human Shape Reconstruction

Nikos Kolotouros, Georgios Pavlakos, Kostas Daniilidis

[conf] [Pytorch code]

HoloPose

HoloPose: Holistic 3D Human Reconstruction In-The-Wild

Riza Alp Guler, Iasonas Kokkinos

[conf] [code]

HMR-video

Learning 3D Human Dynamics From Video

Angjoo Kanazawa, Jason Y. Zhang, Panna Felsen, Jitendra Malik

[conf] [Tensorflow code]

SemGCN

Semantic Graph Convolutional Networks for 3D Human Pose Regression

Long Zhao, Xi Peng, Yu Tian, Mubbasir Kapadia, Dimitris N. Metaxas

[conf] [Pytorch code]

VideoPose3D

3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training

Dario Pavllo, Christoph Feichtenhofer, David Grangier, Michael Auli

[conf] [Pytorch code]

EpipolarPose

Self-Supervised Learning of 3D Human Pose Using Multi-View Geometry

Muhammed Kocabas, Salih Karagoz, Emre Akbas

[conf] [Pytorch code]

2018 ECCV

TP-Net

Exploiting temporal information for 3D human pose estimation

Mir Rayat Imtiaz Hossain, James J. Little

[conf] [code]

IHPR

Integral Human Pose Regression

Xiao Sun, Bin Xiao, Fangyin Wei, Shuang Liang, Yichen Wei

[conf] [Pytorch code]

2018 CVPR

HMR

End-to-End Recovery of Human Shape and Pose

Angjoo Kanazawa, Michael J. Black, David W. Jacobs, Jitendra Malik

[conf] [Tensorflow - official] [Pytorch]

MVC

Learning Monocular 3D Human Pose Estimation From Multi-View Images

Helge Rhodin, Jörg Spörri, Isinsu Katircioglu, Victor Constantin, Frédéric Meyer, Erich Müller, Mathieu Salzmann, Pascal Fua

[conf]

Deephar

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning

Diogo C. Luvizon, David Picard, Hedi Tabia

[conf] [code]

ODS

Ordinal Depth Supervision for 3D Human Pose Estimation
Georgios Pavlakos, Xiaowei Zhou, Kostas Daniilidis

[conf] [code]

Deephar

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning
Diogo C. Luvizon, David Picard, Hedi Tabia

[conf] [code]

2018 Others

LPG-AAAI18

Learning Pose Grammar to Encode Human Body Configuration for 3D Pose Estimation
Hao-Shu Fang, Yuanlu Xu, Wenguan Wang, Xiaobai Liu, Song-Chun Zhu

[conf]

OriNet-BMVC18

OriNet: A Fully Convolutional Network for 3D Human Pose Estimation
Chenxu Luo, Xiao Chu, Alan Yuille
[arxiv] [code]

2017 ICCV

comming soon...

2017 CVPR

UP

Unite the People: Closing the Loop Between 3D and 2D Human Representations

Christoph Lassner, Javier Romero, Martin Kiefel, Federica Bogo, Michael J. Black, Peter V. Gehler

[conf] [code]

RPSM

Recurrent 3D Pose Sequence Machines
Mude Lin, Liang Lin, Xiaodan Liang, Keze Wang, Hui Cheng

[conf] [code]

C2F

Coarse-To-Fine Volumetric Prediction for Single-Image 3D Human Pose
Georgios Pavlakos, Xiaowei Zhou, Konstantinos G. Derpanis, Kostas Daniilidis

[conf] [code]

LFD

Lifting From the Deep: Convolutional 3D Pose Estimation From a Single Image
Denis Tome, Chris Russell, Lourdes Agapito

[conf] [code]

2016 ECCV

SMPLify

Keep it SMPL: Automatic Estimation of {3D} Human Pose and Shape from a Single Image
Bogo Federica, Kanazawa Angjoo, Lassner Christoph, Gehler Peter, Romero Javier, Black Michael J.

[Paper] [code]

2016 CVPR

SMD

Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
Xiaowei Zhou, Menglong Zhu, Spyridon Leonardos, Kosta Derpanis, Kostas Daniilidis

[conf] [code]

[Back to top]

Journal-Papers

FBN

Feature Boosting Network For 3D Pose Estimation
Jun Liu, Henghui Ding, Amir Shahroudy, Ling-Yu Duan, Xudong Jiang, Gang Wang, Alex C. Kot

[arxiv]

Datasets

Deep Fashion3D

Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images
Heming Zhu and Yu Cao and Hang Jin and Weikai Chen and Dong Du and Zhangye Wang and Shuguang Cui and Xiaoguang Han
[arxiv] [project]

3DBodyTex.Pose

Towards Generalization of 3D Human Pose Estimation In The Wild
Renato Baptista, Alexandre Saint, Kassem Al Ismaeil, Djamila Aouada
[arxiv]

GPA

Geometric Pose Affordance: 3D Human Pose with Scene Constraints
Zhe Wang, Liyan Chen, Shaurya Rathore, Daeyun Shin, Charless Fowlkes [arxiv] [project]

JTA

Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World
Matteo Fabbri, Fabio Lanzi, Simone Calderara, Andrea Palazzi, Roberto Vezzani, Rita Cucchiara
[arxiv] [project]

Contribute

Contributions are welcome! Please feel free to pull requests, open an issue or send me email ([email protected]) to add/correct state-of-the-art methods.