Recent Progress in 3D Hand Tasks

A complete paper list for hand pose can be found here. This repo focuses on some subject areas, ideas, and works. Although some papers invloved do not study towards 3D hand scope, they are conceived presenting relavent and inspiring ideas. If there is any mistake or carelessness, please feel free to open an issue or send me email.

Contents

• Contents
• Overview
• Datasets
• Papers
  • Hand Shape/Pose
    • Forward Kinematics
    • Inverse Kinematics
    • Graph
    • Transformer
    • 2.5D
    • Texture
    • Shape
    • Neural Rendering on Generation
    • Neural Rendering on Avatar
    • Dynamics
    • Weak Supervision
    • Temporal
    • Cross-Modal
  • Hand-Object Interaction
    • Kinematics
    • Graph
    • Shape
    • Neural Rendering
    • Dynamics
    • Weak Supervision
    • Temporal
    • Contact
  • Hand-Hand Interaction
    • Kinematics
    • 2.5D
    • Transformer
    • Graph
    • Shape
    • Neural Rendering
    • Gesture Prediction
  • Full Body Reconstruction
    • Forward Kinematics
    • Inverse Kinematics
    • Shape
    • Neural Rendering
    • Grasp
    • In-Sence
• Tools
  • Model
  • MoCap
  • Data

Overview

Datasets

This list presents recent datasets with properties of data size, shape annotation, hand-hand interaction (Hand int.), hand-object interaction (Obj. int.), motion sequence, synthetic, and physical attributes (Phy. att.).

Papers

The below indices are references to the above overview.

Hand Shape/Pose

Forward Kinematics

• [H.FK.14] Overcoming the Trade-off Between Accuracy and Plausibility in 3D Hand Shape Reconstruction. CVPR23. [PDF]
  Ziwei Yu, Chen Li, Linlin Yang, Xiaoxu Zheng, Michael Bi Mi, Gim Hee Lee, Angela Yao

• [H.FK.13] Identity-Aware Hand Mesh Estimation and Personalization from RGB Images. ECCV22. [Code]
  Deying Kong, Linguang Zhang, Liangjian Chen, Haoyu Ma, Xiangyi Yan, Shanlin Sun, Xingwei Liu, Kun Han, Xiaohui Xie

• [H.FK.12] HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Network. CVPR22. [PDF] [Code]
  JoonKyu Park, Yeonguk Oh, Gyeongsik Moon, Hongsuk Choi, Kyoung Mu Lee

• [H.FK.11] Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration. CVPR21. [PDF] [Code]
  Shaofei Wang, Andreas Geiger, Siyu Tang

• [H.FK.10] RealisticHands: A Hybrid Model for 3D Hand Reconstruction. arXiv21. [PDF]
  Michael Seeber, Martin R. Oswald, Roi Poranne

• [H.FK.9] Contrastive Representation Learning for Hand Shape Estimation. arXiv21. [PDF] [Project] [Code]
  Christian Zimmermann, Max Argus, Thomas Brox

• [H.FK.8] Hand Image Understanding via Deep Multi-Task Learning. ICCV21. [PDF]
  Xiong Zhang, Hongsheng Huang, Jianchao Tan, Hongmin Xu, Cheng Yang, Guozhu Peng, Lei Wang, Ji Liu

• [H.FK.7] DeepHandMesh: Weakly-supervised Deep Encoder-Decoder Framework for High-fidelity Hand Mesh Modeling from a Single RGB Image. ECCV20. [PDF] [Project]
  Gyeongsik Moon, Takaaki Shiratori, Kyoung Mu Lee

• [H.FK.6] Knowledge as Priors: Cross-Modal Knowledge Generalization for Datasets without Superior Knowledge. CVPR20. [PDF]
  Long Zhao, Xi Peng, Yuxiao Chen, Mubbasir Kapadia, Dimitris N. Metaxas

• [H.FK.5] FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images. ICCV19. [PDF] [Project]Code]
  Christian Zimmermann, Duygu Ceylan, Jimei Yang, Bryan Russell, Max Argus, Thomas Brox

• [H.FK.4] End-to-end Hand Mesh Recovery from a Monocular RGB Image. ICCV19. [PDF] [Code]
  Xiong Zhang, Qiang Li, Wenbo Zhang, Wen Zheng

• [H.FK.3] 3D Hand Shape and Pose from Images in the Wild. CVPR19. [PDF] [code]
  Adnane Boukhayma, Rodrigo de Bem, Philip H.S. Torr

• [H.FK.2] Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Rendering. CVPR19. [PDF]
  Seungryul Baek, Kwang In Kim, Tae-Kyun Kim

• [H.FK.1] GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB. CVPR18. [PDF] [Supp] [Project]
  Franziska Mueller, Florian Bernard, Oleksandr Sotnychenko, Dushyant Mehta, Srinath Sridhar, Dan Casas, Christian Theobalt

Inverse Kinematics

• [H.IK.4] HandTailor: Towards High-Precision Monocular 3D Hand Recovery. arXiv21. [PDF] [Code]
  Jun Lv, Wenqiang Xu, Lixin Yang, Sucheng Qian, Chongzhao Mao, Cewu Lu

• [H.IK.3] HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation. CVPR21. [PDF] [Code]
  Jiefeng Li, Chao Xu, Zhicun Chen, Siyuan Bian, Lixin Yang, Cewu Lu

• [H.IK.2] BiHand: Recovering Hand Mesh with Multi-stage Bisected Hourglass Networks. BMVC20. [PDF] [Code]
  Lixin Yang, Jiasen Li, Wenqiang Xu, Yiqun Diao, Cewu Lu

• [H.IK.1] Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data. CVPR20. [PDF] [Project] [Code]
  Yuxiao Zhou, Marc Habermann, Weipeng Xu, Ikhsanul Habibie, Christian Theobalt, Feng Xu

Graph

• [H.GH.11] High Fidelity 3D Hand Shape Reconstruction via Scalable Graph Frequency Decomposition. CVPR23. [PDF]
  Tianyu Luan, Yuanhao Zhai, Jingjing Meng, Zhong Li, Zhang Chen, Yi Xu, and Junsong Yuan

• [H.GH.10] MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Image. CVPR22. [PDF] [Code]
  Xingyu Chen, Yufeng Liu, Yajiao Dong, Xiong Zhang, Chongyang Ma, Yanmin Xiong, Yuan Zhang, Xiaoyan Guo

• [H.GH.9] JGR-P2O: Joint Graph Reasoning based Pixel-to-Offset Prediction Network for 3D Hand Pose Estimation from a Single Depth Image. ECCV20 [PDF] [Code]
  Linpu Fang, Xingyan Liu, Li Liu, Hang Xu, Wenxiong Kang

• [H.GH.8] Point2Skeleton: Learning Skeletal Representations from Point Clouds. CVPR21. [PDF] [Code]
  Cheng Lin, Changjian Li, Yuan Liu, Nenglun Chen, Yi-King Choi, Wenping Wang

• [H.GH.7] DC-GNet: Deep Mesh Relation Capturing Graph Convolution Network for 3D Human Shape Reconstruction, MM21. [PDF]
  Shihao Zhou, Mengxi Jiang, Shanshan Cai, Yunqi Lei

• [H.GH.6] MVHM: A Large-Scale Multi-View Hand Mesh Benchmark for Accurate 3D Hand Pose Estimation. WACV21. [PDF]
  Liangjian Chen, Shih-Yao Lin, Yusheng Xie, Yen-Yu Lin, Xiaohui Xie

• [H.GH.5] Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registration. CVPR21. [PDF] [Code]
  Xingyu Chen, Yufeng Liu, Chongyang Ma, Jianlong Chang, Huayan Wang, Tian Chen, Xiaoyan Guo, Pengfei Wan, Wen Zheng

• [H.GH.4] Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose. ECCV20. [PDF] [Code]
  Hongsuk Choi, Gyeongsik Moon, Kyoung Mu Lee

• [H.GH.3] Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild. CVPR20. [PDF] [Project]
  Dominik Kulon, Riza Alp Güler, Iasonas Kokkinos, Michael Bronstein, Stefanos Zafeiriou

• [H.GH.2] Exploiting Spatial-temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks. ICCV19. [PDF]
  Yujun Cai, Liuhao Ge, Jun Liu, Jianfei Cai, Tat-Jen Cham, Junsong Yuan, and Nadia Magnenat Thalmann

• [H.GH.1] 3D Hand Shape and Pose Estimation from a Single RGB Image. CVPR19. [PDF] [Project] [Code]
  Liuhao Ge, Zhou Ren, Yuncheng Li, Zehao Xue, Yingying Wang, Jianfei Cai, Junsong Yuan

Transformer

• [H.TR.9] POEM: Reconstructing Hand in a Point Embedded Multi-view Stereo. CVPR23. [PDF] [Code]
  Lixin Yang, Jian Xu, Licheng Zhong, Xinyu Zhan, Zhicheng Wang, Kejian Wu, Cewu Lu

• [H.TR.8] Recovering 3D Hand Mesh Sequence from a Single Blurry Image: A New Dataset and Temporal Unfolding. CVPR23. [PDF] [Code]
  Yeonguk Oh, JoonKyu Park, Jaeha Kim, Gyeongsik Moon, Kyoung Mu Lee

• [H.TR.7] Deformer: Dynamic Fusion Transformer for Robust Hand Pose Estimation. [PDF]
  Qichen Fu,Xingyu Liu,Ran Xu,Juan Carlos Niebles,Kris M. Kitani

• [H.TR.6] Spatial-Temporal Parallel Transformer for Arm-Hand Dynamic Estimation. CVPR22. [PDF]
  Shuying Liu, Wenbin Wu, Jiaxian Wu, Yue Lin

• [H.TR.5] GraFormer: Graph Convolution Transformer for 3D Pose Estimation. arXiv21. [PDF] [Code]
  Weixi Zhao, Yunjie Tian, Qixiang Ye, Jianbin Jiao and Weiqiang Wang

• [H.TR.4] Mesh Graphormer. ICCV21. [PDF] [Code]
  Kevin Lin, Lijuan Wang, Zicheng Liu

• [H.TR.3] End-to-End Human Pose and Mesh Reconstruction with Transformers. CVPR21. [PDF] [Code]
  Kevin Lin, Lijuan Wang, Zicheng Liu

• [H.TR.2] Hand-Transformer: Non-Autoregressive Structured Modeling for 3D Hand Pose Estimation. ECCV20. [PDF]
  Lin Huang, Jianchao Tan, Ji Liu, and Junsong Yuan

• [H.TR.1] Epipolar Transformers. CVPR20. [PDF] [Code]
  Yihui He, Rui Yan, Shoou-I Yu, Katerina Fragkiadaki

2.5D

• [H.VX.7] Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction. ICCV21. [PDF] [Code]
  Xiao Tang, Tianyu Wang, Chi-Wing Fu

• [H.VX.6] Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild. CVPR20. [PDF]
  Umar Iqbal, Pavlo Molchanov, Jan Kautz

• [H.VX.5] HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks. arXiv21. [PDF]
  Jameel Malik, Soshi Shimada, Ahmed Elhayek, Sk Aziz Ali, Christian Theobalt, Vladislav Golyanik, Didier Stricker

• [H.VX.4] Exploiting Learnable Joint Groups for Hand Pose Estimation. AAAI21. [PDF] [Code] Moran Li, Yuan Gao, Nong Sang

• [H.VX.3] I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image. ECCV20. [PDF] [Code]
  Gyeongsik Moon, Kyoung Mu Lee

• [H.VX.2] HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map. CVPR20. [PDF]
  Jameel Malik, Ibrahim Abdelaziz, Ahmed Elhayek, Soshi Shimada, Sk Aziz Ali, Vladislav Golyanik, Christian Theobalt, Didier Stricker

• [H.VX.1] Hand Pose Estimation via Latent 2.5D Heatmap Regression. ECCV18. [PDF]
  Umar Iqbal, Pavlo Molchanov, Thomas Breuel, Juergen Gall, Jan Kautz

Texture

• [H.UV.6] A Probabilistic Attention Model with Occlusion-aware Texture Regression for 3D Hand Reconstruction from a Single RGB Image. CVPR23. [PDF] [Code]
  Zheheng Jiang, Hossein Rahmani, Sue Black, Bryan M. Williams

• [H.UV.5] Handy: Towards a high fidelity 3D hand shape and appearance model. CVPR23. [PDF] [Code]
  Rolandos Potamias Potamias, Stylianos Ploumpis, Stylianos Moschoglou, Vasileios Triantafyllou, Stefanos Zafeiriou

• [H.UV.4] Semi-supervised Hand Appearance Recovery via Structure Disentanglement and Dual Adversarial Discrimination. CVPR23. [PDF]
  Zimeng Zhao, Binghui Zuo, Zhiyu Long, Yangang Wang

• [H.UV.3] DART: Articulated Hand Model with Diverse Accessories and Rich Textures. NeurIPS22. [PDF] [Project]
  Daiheng Gao, Yuliang Xiu, Kailin Li, Lixin Yang, Feng Wang, Peng Zhang, Bang Zhang, Cewu Lu, Ping Tan

• [H.UV.2] I2UV-HandNet: Image-to-UV Prediction Network for Accurate and High-fidelity 3D Hand Mesh Modeling. ICCV21. [PDF]
  Ping Chen, Yujin Chen, Dong Yang, Fangyin Wu, Qin Li, Qingpei Xia, Yong Tan

• [H.UV.1] HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalizationm. ECCV20. [PDF] [Project]
  Neng Qian, Jiayi Wang, Franziska Mueller, Florian Bernard, Vladislav Golyanik, Christian Theobalt

Shape

• [H.SP.6] Neural Voting Field for Camera-Space 3D Hand Pose Estimation. CVPR23. [PDF] [Project]
  Lin Huang, Chung-Ching Lin, Kevin Lin, Lin Liang, Lijuan Wang, Junsong Yuan, Zicheng Liu

• [H.SP.5] COAP: Compositional Articulated Occupancy of People. CVPR22. [PDF] [Project] [Code]
  Marko Mihajlovic, Shunsuke Saito, Aayush Bansal, Michael Zollhoefer, Siyu Tang

• [H.SP.4] A Skeleton-Driven Neural Occupancy Representation for Articulated Hands. 3DV21. [PDF]
  Korrawe Karunratanakul, Adrian Spurr, Zicong Fan, Otmar Hilliges, Siyu Tang

• [H.SP.3] SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes. ICCV21. [PDF] [Code] [Project]
  Xu Chen, Yufeng Zheng, Michael J. Black, Otmar Hilliges, Andreas Geiger

• [H.SP.2] LEAP: Learning Articulated Occupancy of People. CVPR21 [PDF] [Code]
  Marko Mihajlovic, Yan Zhang, Michael J. Black, Siyu Tang

• [H.SP.1] NASA: Neural Articulated Shape Approximation. ECCV20. [PDF] [Project]
  Boyang Deng, JP Lewis, Timothy Jeruzalski, Gerard Pons-Moll, Geoffrey Hinton, Mohammad Norouzi, Andrea Tagliasacchi

Neural Rendering on Generation

• [H.GEN.8] AG3D: Learning to Generate 3D Avatars from 2D Image Collections. arXiv23. [PDF] [Project] [Code]
  Zijian Dong, Xu Chen, Jinlong Yang3, Michael J. Black3, Otmar Hilliges1, Andreas Geiger

• [H.GEN.7] Generative Neural Articulated Radiance Fields. NeurIPS22. [PDF] [Project]
  Alexander W. Bergman, Petr Kellnhofer, Wang Yifan, Eric R. Chan, David B. Lindell, Gordon Wetzstein

• [H.GEN.6] 3DHumanGAN: Towards Photo-Realistic 3D-Aware Human Image Generation. arXiv22. [PDF] [Project] [Code]
  Zhuoqian Yang, Shikai Li, Wayne Wu, Bo Dai

• [H.GEN.5] HumanGen: Generating Human Radiance Fields with Explicit Priors. arXiv22. [PDF]
  Suyi Jiang, Haoran Jiang, Ziyu Wang, Haimin Luo, Wenzheng Chen, Lan Xu

• [H.GEN.4] AvatarGen: a 3D Generative Model for Animatable Human Avatars. arXiv22. [PDF] [Code]
  Jianfeng Zhang, Zihang Jiang, Dingdong Yang, Hongyi Xu, Yichun Shi, Guoxian Song, Zhongcong Xu, Xinchao Wang, Jiashi Feng

• [H.GEN.3] Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars. arXiv22. [PDF] [Project] [Code]
  Jingxiang Sun, Xuan Wang, Lizhen Wang, Xiaoyu Li, Yong Zhang, Hongwen Zhang, Yebin Liu

• [H.GEN.2] EVA3D: Compositional 3D Human Generation from 2D Image Collections. ICLR23. [PDF] [Project] [Code]
  Fangzhou Hong, Zhaoxi Chen, Yushi Lan, Liang Pan, Ziwei Liu

• [H.GEN.1] Efficient Geometry-aware 3D Generative Adversarial Networks. CVPR22. [PDF] [Project] [Code]
  Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein

Neural Rendering on Avatar

• [H.AVA.24] HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion. arXiv23. [PDF] Mustafa Işık, Martin Rünz, Markos Georgopoulos, Taras Khakhulin, Jonathan Starck, Lourdes Agapito, Matthias Nießner

• [H.AVA.23] Structured 3D Features for Reconstructing Relightable and Animatable Avatars. CVPR23. [PDF] [Project]
  Enric Corona, Mihai Zanfir, Thiemo Alldieck, Eduard Gabriel Bazavan, Andrei Zanfir, Cristian Sminchisescu

• [H.AVA.22] SHERF: Generalizable Human NeRF from a Single Image. arXiv23. [PDF] [Project]
  Shoukang Hu, Fangzhou Hong, Liang Pan, Haiyi Mei, Lei Yang, Ziwei Liu

• [H.AVA.21] RelightableHands: Efficient Neural Relighting of Articulated Hand Models. CVPR23. [PDF] [Project]
  Shun Iwase, Shunsuke Saito, Tomas Simon, Stephen Lombardi, Timur Bagautdinov, Rohan Joshi, Fabian Prada, Takaaki Shiratori, Yaser Sheikh, Jason Saragih

• [H.AVA.20] HARP: Personalized Hand Reconstruction from a Monocular RGB Video. CVPR23. [PDF]
  Korrawe Karunratanakul, Sergey Prokudin, Otmar Hilliges, Siyu Tang

• [H.AVA.19] One-shot Implicit Animatable Avatars with Model-based Priors. arXiv22. [PDF]
  Yangyi Huang, Hongwei Yi, Weiyang Liu, Haofan Wang, Boxi Wu, Wenxiao Wang, Binbin Lin, Debing Zhang, Deng Cai

• [H.AVA.18] NDF: Neural Deformable Fields for Dynamic Human Modelling. ECCV22. [PDF]
  Ruiqi Zhang, Jie Chen

• [H.AVA.17] ARAH: Animatable Volume Rendering of Articulated Human SDFs. ECCV22. [PDF] [Project] [Code]
  Shaofei Wang, Katja Schwarz, Andreas Geiger, Siyu Tang

• [H.AVA.16] Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Video. CVPR23. [PDF] [Project]
  Xingyu Chen, Baoyuan Wang Heung-Yeung, Shum

• [H.AVA.15] Authentic Volumetric Avatars From a Phone Scan. SIGGRAPH22. [PDF]
  Chen Cao, Tomas Simon, Jin Kyu Kim, Gabe Schwartz, Michael Zollhoefer, Shunsuke Saito, Stephen Lombardi, Shih-En Wei, Danielle Belko, Shoou-I Yu, Yaser Sheikh, Jason Saragih

• [H.AVA.14] Mixture of volumetric primitives for efficient neural rendering. SIGGRAPH21. [PDF] [Code]
  Stephen Lombardi, Tomas Simon, Gabriel Schwartz, Michael Zollhoefer, Yaser Sheikh, Jason Saragih

• [H.AVA.13] I M Avatar: Implicit Morphable Head Avatars from Videos. CVPR22. [PDF] [Project] [Code]
  Yufeng Zheng, Victoria Fernández Abrevaya, Marcel C. Bühler, Xu Chen, Michael J. Black, Otmar Hilliges

• [H.AVA.12] Neural Head Avatars From Monocular RGB Videos. CVPR22. [PDF] [Project] [Code]
  Philip-William Grassal, Malte Prinzler, Titus Leistner, Carsten Rother, Matthias Nießner, Justus Thies

• [H.AVA.11] LISA: Learning Implicit Shape and Appearance of Hands. CVPR22. [PDF]
  Enric Corona, Tomas Hodan, Minh Vo, Francesc Moreno-Noguer, Chris Sweeney, Richard Newcombe, Lingni Ma

• [H.AVA.10] Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis. CVPR22. [PDF] [Project] [Code]
  Tianhan Xu, Yasuhiro Fujita, Eiichi Matsumoto

• [H.AVA.9] Structured Local Radiance Fields for Human Avatar Modeling. CVPR22. [PDF] [Project]
  Zerong Zheng, Han Huang, Tao Yu, Hongwen Zhang, Yandong Guo, Yebin Liu

• [H.AVA.8] Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control. SIGGRAPHAsia21. [PDF] [Project]
  Lingjie Liu, Marc Habermann, Viktor Rudnev, Kripasindhu Sarkar, Jiatao Gu, Christian Theobalt

• [H.AVA.7] Editable Free-viewpoint Video Using a Layered Neural Representation. SIGGRAPH21. [PDF] [Project] [Code]
  Jiakai Zhang, Xinhang Liu, Xinyi Ye, Fuqiang Zhao, Yanshun Zhang, Minye Wu, Yingliang Zhang, Lan Xu, Jingyi Yu

• [H.AVA.6] NeuMan: Neural Human Radiance Field from a Single Video. arXiv22. [PDF]
  Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, Anurag Ranjan

• [H.AVA.5] HumanNeRF: Generalizable Neural Human Radiance Field from Sparse Inputs. CVPR22. [PDF] [Project]
  Fuqiang Zhao, Wei Yang, Jiakai Zhang, Pei Lin, Yingliang Zhang, Jingyi Yu, Lan Xu

• [H.AVA.4] HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video. CVPR22. [PDF] [Project]
  Chung-Yi Weng, Brian Curless, Pratul P. Srinivasan, Jonathan T. Barron, Ira Kemelmacher-Shlizerman

• [H.AVA.3] Vid2Actor: Free-viewpoint Animatable Person Synthesis from Video in the Wild. arXiv20. [PDF] [Project]
  Chung-Yi Weng, Brian Curless, Ira Kemelmacher-Shlizerman

• [H.AVA.2] Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies. ICCV21 [PDF] [Code] [Project]
  Sida Peng, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Xiaowei Zhou, Hujun Bao

• [H.AVA.1] Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans. CVPR21. [PDF] [Code] [Project]
  Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, Xiaowei Zhou

Dynamics

• [H.DN.5] PhysDiff: Physics-Guided Human Motion Diffusion Model. arXiv22. [PDF] [Project]
  Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, Jan Kautz

• [H.DN.4] Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capture. CVPR22. [PDF] [Project]
  Buzhen Huang, Liang Pan, Yuan Yang, Jingyi Ju, Yangang Wang

• [H.DN.3] PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision. CVPR22. [PDF] [Code]
  Kehong Gong, Bingbing Li, Jianfeng Zhang, Tao Wang, Jing Huang, Michael Bi Mi, Jiashi Feng, Xinchao Wang

• [H.DN.2] SimPoE: Simulated Character Control for 3D Human Pose Estimation. ICCV21. [PDF] [Project]
  Ye Yuan, Shih-En Wei, Tomas Simon, Kris Kitani, Jason Saragih

• [H.DN.1] Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis. NeurIPS20. [PDF] [Project] [Code]
  Ye Yuan, Kris Kitani

Weak Supervision

• [H.WS.17] CycleHand: Increasing 3D Pose Estimation Ability on In-the-wild Monocular Image through Cyclic Flow. MM22. [PDF]
  Daiheng Gao, Xindi Zhang, Xingyu Chen, Andong Tan, Bang Zhang, Pan Pan, Ping Tan

• [H.WS.16] Image-free Domain Generalization via CLIP for 3D Hand Pose Estimation. arXiv22. [PDF]
  Seongyeong Lee, Hansoo Park, Dong Uk Kim, Jihyeon Kim, Muhammadjon Boboev, Seungryul Baek

• [H.WS.15] Domain Adaptive Hand Keypoint and Pixel Localization in the Wild. ECCV22. [PDF] [Project]
  Takehiko Ohkawa, Yu-Jhe Li, Qichen Fu, Ryosuke Furuta, Kris M. Kitani, and Yoichi Sato

• [H.WS.14] End-to-end Weakly-supervised Multiple 3D Hand Mesh Reconstruction from Single Image. arXiv22. [PDF]
  Jinwei Ren, Jianke Zhu, Jialiang Zhang

• [H.WS.13] Consistent 3D Hand Reconstruction in Video via Self-Supervised Learning. arXiv22. [PDF]
  Zhigang Tu, Zhisheng Huang, Yujin Chen, Di Kang, Linchao Bao, Bisheng Yang, Junsong Yuan

• [H.WS.12] Weakly-supervised 3D Hand Pose Estimation from Monocular RGB Images. ECCV18. [PDF]
  Yujun Cai, Liuhao Ge, Jianfei Cai, and Junsong Yuan

• [H.WS.11] Aligning Latent Spaces for 3D Hand Pose Estimation. ICCV19. [PDF]
  Linlin Yang, Shile Li, Dongheui Lee, Angela Yao

• [H.WS.10] Semi-Supervised 3D Hand Shape and Pose Estimation with Label Propagation. arXiv21. [PDF]
  Samira Kaviani, Amir Rahimi, Richard Hartley

• [H.WS.9] Self-supervised Transfer Learning for Hand Mesh Recovery from Binocular Images. ICCV21. [PDF]
  Zheng Chen, Sihan Wang, Yi Sun, Xiaohong Ma

• [H.WS.8] SO-HandNet: Self-Organizing Network for 3D Hand Pose Estimation with Semi-supervised Learning. ICCV19. [PDF]
  Yujin Chen, Zhigang Tu, Liuhao Ge, Dejun Zhang, Ruizhi Chen, Junsong Yuan

• [H.WS.7] PeCLR: Self-Supervised 3D Hand Pose Estimation from Monocular RGB via Contrastive Learning. ICCV21. [PDF] [Project] [Code]
  Adrian Spurr, Aneesh Dahiya, Xucong Zhang, Xi Wang, Otmar Hilliges

• [H.WS.6] Adversarial Motion Modelling helps Semi-supervised Hand Pose Estimation. arXiv21. [PDF]
  Adrian Spurr, Pavlo Molchanov, Umar Iqbal, Jan Kautz, Otmar Hilliges

• [H.WS.5] SemiHand: Semi-supervised Hand Pose Estimation with Consistency. ICCV21. [PDF]
  Linlin Yang, Shicheng Chen, Angela Yao

• [H.WS.4] S2HAND: Model-based 3D Hand Reconstruction via Self-Supervised Learning. CVPR21. [PDF] [Project] [Code]
  Yujin Chen, Zhigang Tu, Di Kang, Linchao Bao, Ying Zhang, Xuefei Zhe, Ruizhi Chen, Junsong Yuan

• [H.WS.3] Weakly-Supervised 3D Hand Pose Estimation via Biomechanical Constraints. ECCV20. [PDF]
  Adrian Spurr, Umar Iqbal, Pavlo Molchanov, Otmar Hilliges, Jan Kautz

• [H.WS.2] Dual Grid Net: hand mesh vertex regression from single depth maps. ECCV20. [PDF]
  Chengde Wan, Thomas Probst, Luc Van Gool, Angela Yao

• [H.WS.1] Adaptive Wasserstein Hourglass for Weakly Supervised Hand Pose Estimation from Monocular RGB. MM20. [PDF]
  Yumeng Zhang, Li Chen, Yufeng Liu, Junhai Yong, Wen Zheng

Temporal

• [H.TP.9] Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videos. CVPR23. [PDF] [Project] [Code]
  Yilin Wen, Hao Pan, Lei Yang, Jia Pan, Taku Komura, Wenping Wang

• [H.TP.8] Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation from Monocular Video. CVPR22. [PDF] [Project]
  Wen-Li Wei, Jen-Chun Lin, Tyng-Luh Liu, Hong-Yuan Mark Liao

• [H.TP.7] SmoothNet: A Plug-and-Play Network for Refining Human Poses in Videos. arXiv21. [PDF]
  Ailing Zeng, Lei Yang, Xuan Ju, Jiefeng Li, Jianyi Wang, Qiang Xu

• [H.TP.6] Multi-View Video-Based 3D Hand Pose Estimation. arXiv21. [PDF] [Code]
  Leyla Khaleghi, Alireza Sepas Moghaddam, Joshua Marshall, Ali Etemad

• [H.TP.5] TravelNet: Self-supervised Physically Plausible Hand Motion Learning from Monocular Color Images. ICCV21. [PDF]
  Zimeng Zhao, Xi Zhao and Yangang Wang

• [H.TP.4] Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video. CVPR21. [PDF] [Code]
  Hongsuk Choi, Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee

• [H.TP.3] Temporal-Aware Self-Supervised Learning for 3D Hand Pose and Mesh Estimation in Videos. WACV21. [PDF]
  Liangjian Chen, Shih-Yao Lin, Yusheng Xie, Yen-Yu Lin, Xiaohui Xie

• [H.TP.2] Adaptive Computationally Efficient Network for Monocular 3D Hand Pose Estimation. ECCV20. [PDF]
  Zhipeng Fan, Jun Liu, Yao Wang

• [H.TP.1] SeqHAND: RGB-Sequence-Based 3D Hand Pose and Shape Estimation. ECCV20. [PDF]
  John Yang, Hyung Jin Chang, Seungeui Lee, Nojun Kwak

Cross-Modal

• [H.CM.3] Cross-domain 3D Hand Pose Estimation with Dual Modalities. CVPR23. [PDF]
  Qiuxia Lin, Linlin Yang, Angela Yao

• [H.CM.2] Knowledge as Priors: Cross-Modal Knowledge Generalization for Datasets without Superior Knowledge. CVPR20. [PDF]
  Long Zhao, Xi Peng, Yuxiao Chen, Mubbasir Kapadia, Dimitris N. Metaxas

• [H.CM.1] Cross-modal Deep Variational Hand Pose Estimation. CVPR18. [PDF]
  Adrian Spurr, Jie Song, Seonwook Park, Otmar Hilliges

[back to top]

Hand-Object Interaction

Kinematics

• [HO.FK.10] Harmonious Feature Learning for Interactive Hand-Object Pose Estimation. CVPR23. [PDF] [Code]
  Zhifeng Lin, Changxing Ding, Huan Yao, Zengsheng Kuang, Shaoli Huang

• [HO.FK.9] H2O: A Benchmark for Visual Human-human Object Handover Analysis. ICCV21. [PDF]
  Ruolin Ye, Wenqiang Xu, Zhendong Xue, Tutian Tang, Yanfeng Wang, Cewu Lu

• [HO.FK.8] ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis. CVPR22. [PDF] [Code]
  Kailin Li, Lixin Yang, Xinyu Zhan, Jun Lv, Wenqiang Xu, Jiefeng Li, Cewu Lu

• [HO.FK.7] OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction. CVPR22. [PDF] [Code]
  Lixin Yang, Kailin Li Xinyu Zhan, Fei Wu, Anran Xu, Liu Liu, Cewu Lu

• [HO.FK.6] CPF: Learning a Contact Potential Field to Model the Hand-object Interaction. ICCV21. [PDF] [Code]
  Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Jiefeng Li, Cewu Lu

• [HO.FK.5] HO-3D_v3: Improving the Accuracy of Hand-Object Annotations of the HO-3D Dataset. arXiv21. [PDF]
  Shreyas Hampali, Sayan Deb Sarkar, Vincent Lepetit

• [HO.FK.4] Unsupervised Domain Adaptation with Temporal-Consistent Self-Training for 3D Hand-Object Joint Reconstruction. arXiv21. [PDF]
  Mengshi Qi, Edoardo Remelli, Mathieu Salzmann, Pascal Fua

• [HO.FK.3] DexYCB: A Benchmark for Capturing Hand Grasping of Objects. CVPR21. [PDF] [Project] [Code]
  Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj S. Narang, Karl Van Wyk, Umar Iqbal, Stan Birchfield, Jan Kautz, Dieter Fox

• [HO.FK.2] HOnnotate: A method for 3D Annotation of Hand and Objects Poses. CVPR20. [PDF] [Project] [Code]
  Shreyas Hampali, Mahdi Rad, Markus Oberweger, Vincent Lepetit

• [HO.FK.1] Learning joint reconstruction of hands and manipulated objects. CVPR19. [PDF] [Project] [Code]
  Yana Hasson, Gül Varol, Dimitris Tzionas, Igor Kalevatykh, Michael J. Black, Ivan Laptev, and Cordelia Schmid

Graph

• [HO.GH.7] THOR-Net: End-to-end Graformer-based Realistic Two Hands and Object Reconstruction with Self-supervision. WACV23. [PDF] [Code]
  Ahmed Tawfik Aboukhadra, Jameel Malik, Ahmed Elhayek, Nadia Robertini, Didier Stricker

• [HO.GH.6] Interacting Hand-Object Pose Estimation via Dense Mutual Attention. WACV23. [PDF] [Code]
  Rong Wang Wei Mao Hongdong Li

• [HO.GH.5] Scene Synthesis from Human Motion. SIGGRAPH Asia22. [PDF]
  Sifan Ye, Yixing Wang, Jiaman Li, Dennis Park, C. Karen Liu, Huazhe Xu, Jiajun Wu

• [HO.GH.4] S2Contact: Graph-based Network for 3D Hand-Object Contact Estimation with Semi-Supervised Learning. ECCV22. [PDF] [Project] [Code]
  Takehiko Ohkawa, Yu-Jhe Li, Qichen Fu, Ryosuke Furuta, Kris M. Kitani, Yoichi Sato

• [HO.GH.3] Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolution. CVPR22. [PDF]
  Tze Ho Elden Tse, Kwang In Kim, Ales Leonardis, Hyung Jin Chang

• [HO.GH.2] H2O: Two Hands Manipulating Objects for First Person Interaction Recognition. ICCV21. [PDF] [Project] [Code]
  Taein Kwon, Bugra Tekin, Jan Stuhmer, Federica Bogo, Marc Pollefeys

• [HO.GH.1] HOPE-Net: A Graph-based Model for Hand-Object Pose Estimation. CVPR20. [PDF] [Code]
  Bardia Doosti, Shujon Naha, Majid Mirbagheri, David Crandall

Shape

• [HO.SP.4] gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction. CVPR23. [PDF] [Project]
  Zerui Chen, Shizhe Chen, Cordelia Schmid, Ivan Laptev

• [HO.SP.3] AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstruction. ECCV22. [PDF] [Project] [Code]
  Zerui Chen, Yana Hasson, Cordelia Schmid, Ivan Laptev

• [HO.SP.2] What’s in your hands? 3D Reconstruction of Generic Objects in Hands. CVPR22. [PDF] [Project] [Code]
  Yufei Ye, Abhinav Gupta, Shubham Tulsiani

• [HO.SP.1] Grasping Field: Learning Implicit Representations for Human Grasps. 3DV20. [PDF] [Code]
  Korrawe Karunratanakul, Jinlong Yang, Yan Zhang, Michael Black, Krikamol Muandet, Siyu Tang

Neural Rendering

• [HO.NR.5] Affordance Diffusion: Synthesizing Hand-Object Interactions. CVPR23. [PDF] [Project]
  Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song, Shubham Tulsiani, Sifei Liu

• [HO.NR.4] Hand-Object Interaction Image Generation. NeurIPS22. [PDF] [Project]
  Hezhen Hu, Weilun Wang, Wengang Zhou, Houqiang Li

• [HO.NR.3] In-Hand 3D Object Scanning from an RGB Sequence. arXiv22. [PDF]
  Shreyas Hampali, Tomas Hodan, Luan Tran, Lingni Ma, Cem Keskin, Vincent Lepetit

• [HO.NR.2] Reconstructing Hand-Held Objects from Monocular Video. SIGGRAPH Asia22. [PDF] [Project]
  Di Huang,Xiaopeng Ji,Xingyi He,Jiaming Sun,Tong He,Qing Shuai,Wanli Ouyang,Xiaowei Zhou

• [HO.NR.1] NeuralHOFusion: Neural Volumetric Rendering Under Human-Object Interactions. CVPR22. [PDF]
  Yuheng Jiang, Suyi Jiang, Guoxing Sun, Zhuo Su, Kaiwen Guo, Minye Wu, Jingyi Yu, Lan Xu

Dynamics

• [HO.DN.3] D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions. CVPR22. [PDF] [Project]
  Sammy Christen, Muhammed Kocabas, Emre Aksan, Jemin Hwangbo, Jie Song, Otmar Hilliges

• [HO.DN.2] DexMV: Imitation Learning for Dexterous Manipulation from Human Videos. ICCVW21. [PDF] [Project]
  Yuzhe Qin, Yueh-Hua Wu, Shaowei Liu, Hanwen Jiang, Ruihan Yang, Yang Fu, Xiaolong Wang

• [HO.DN.1] Stability-driven Contact Reconstruction From Monocular Color Images. CVPR22. [PDF]
  Zimeng Zhao, Binghui Zuo, Wei Xie, Yangang Wang

Weak Supervision

• [HO.WS.2] Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time. CVPR21. [PDF] [Project] [Code]
  Shaowei Liu, Hanwen Jiang*, Jiarui Xu, Sifei Liu, Xiaolong Wang*

• [HO.WS.1] Weakly-supervised Domain Adaptation via GAN and Mesh Model for Estimating 3D Hand Poses Interacting Objects. CVPR20. [PDF] [Code]
  Seungryul Baek, Kwang In Kim, Tae-Kyun Kim

Temporal

• [HO.TP.2] Towards unconstrained joint hand-object reconstruction from RGB videos. arXiv21. [PDF] [Project] [Code]
  Yana Hasson, Gül Varol, Ivan Laptev, Cordelia Schmid

• [HO.TP.1] Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction. CVPR20. [PDF] [Project] [Code]
  Yana Hasson, Bugra Tekin, Federica Bogo, Ivan Laptev, Marc Pollefeys, Cordelia Schmid

Contact

• [HO.Con.10] Transformer-based Unified Recognition of Two Hands Manipulating Objects. CVPR23. [PDF] Hoseong Cho, Chanwoo Kim, Jihyeon Kim, Seongyeong Lee, Elkhan Ismayilzada, Seungryul Baek

• [HO.Con.9] Visual-Tactile Sensing for In-Hand Object Reconstruction. CVPR23. [PDF] [Project]
  Wenqiang Xu, Zhenjun Yu, Han Xue, Ruolin Ye, Siqiong Yao, Cewu Lu

• [HO.Con.8] UV-Based 3D Hand-Object Reconstruction with Grasp Optimization. BMVC22. [PDF]
  Ziwei Yu,Linlin Yang,You Xie,Ping Chen,Angela Yao

• [HO.Con.7] ContactOpt: Optimizing Contact to Improve Grasps. CVPR21. [PDF] [Code]
  Patrick Grady, Chengcheng Tang, Christopher D. Twigg, Minh Vo, Samarth Brahmbhatt, Charles C. Kemp

• [HO.Con.6] CHORE: Contact, Human and Object REconstruction from a single RGB image. arXiv22. [PDF]
  Xianghui Xie, Bharat Lal Bhatnagar, Gerard Pons-Moll

• [HO.Con.5] Understanding Egocentric Hand-Object Interactions from Hand Pose Estimation. arXiv21. [PDF]
  Yao Lu, Walterio W. Mayol-Cuevas

• [HO.Con.4] Reconstructing Hand-Object Interactions in the Wild. ICCV21. [PDF] [Code]
  Zhe Cao, Ilija Radosavovic*, Angjoo Kanazawa, Jitendra Malik*

• [HO.Con.3] Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild. ECCV20. [PDF] [Code]
  Jason Y. Zhang, Sam Pepose, Hanbyul Joo, Deva Ramanan, Jitendra Malik, Angjoo Kanazawa

• [HO.Con.2] GanHand: Predicting Human Grasp Affordances in Multi-Object Scenes. CVPR20. [PDF] [Code]
  Enric Corona, Albert Pumarola, Guillem Alenya, Francesc Moreno-Noguer, Gregory Rogez

• [HO.Con.1] Hand-Object Contact Consistency Reasoning for Human Grasps Generation. ICCV21. [PDF] [Project]
  Hanwen Jiang, Shaowei Liu, Jiashun Wang, Xiaolong Wang

[back to top]

Hand-Hand Interaction

Kinematics

• [HH.FK.6] ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction. CVPR23. [PDF] [Code]
  Zhengdi Yu,Shaoli Huang,Chen Fang,Toby P. Breckon,Jue Wang

• [HH.FK.5] UmeTrack: Unified multi-view end-to-end hand tracking for VR. SIGGRAPH Asia 2022. [PDF]
  Shangchen Han, Po-chen Wu, Yubo Zhang, Beibei Liu, Linguang Zhang, Zheng Wang, Weiguang Si, Peizhao Zhang, Yujun Cai, Tomas Hodan, Randi Cabezas, Luan Tran, Muzaffer Akbay, Tsz-Ho Yu, Cem Keskin, Robert Wang

• [HH.FK.4] Constraining Dense Hand Surface Tracking with Elasticity. TOG20. [PDF]
  Breannan Smith, Chenglei Wu, He Wen, Patrick Peluse, Yaser Sheikh, Jessica Hodgins, Takaaki Shiratori

• [HH.FK.3] Coherent Reconstruction of Multiple Humans from a Single Image. CVPR20 [PDF] [Project] [Code]
  Wen Jiang, Nikos Kolotouros, Georgios Pavlakos, Xiaowei Zhou, Kostas Daniilidis

• [HH.FK.2] Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements. 3DV21. [PDF] [Project] [Code]
  Yu Rong, Jingbo Wang, Ziwei Liu, Chen Change Loy

• [HH.FK.1] RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video. SIGGRAPHAsia20 [PDF] [Project]
  Jiayi Wang, Franziska Mueller, Florian Bernard, Suzanne Sorli, Oleksandr Sotnychenko, Neng Qian, Miguel A. Otaduy, Dan Casas, and Christian Theobalt

2.5D

• [HH.VX.5] Bringing Inputs to Shared Domains for 3D Interacting Hands Recovery in the Wild. CVPR23. [PDF] [Code]
  Gyeongsik Moon

• [HH.VX.4] End-toend detection and pose estimation of two interacting hands. ICCV21. [PDF]
  Dong Uk Kim, Kwang In Kim, Seungryul Baek

• [HH.VX.3] Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentation. 3DV21. [PDF]
  Zicong Fan, Adrian Spurr, Muhammed Kocabas, Siyu Tang, Michael J. Black, Otmar Hilliges

• [HH.VX.2] Interacting Two-Hand 3D Pose and Shape Reconstruction from Single Color Image. ICCV21. [PDF] [Project] [Code]
  Baowen Zhang, Yangang Wang, Xiaoming Deng, Yinda Zhang, Ping Tan, Cuixia Ma and Hongan Wang

• [HH.VX.1] InterHand2.6M: A New Large-scale Dataset and Baseline for 3D Single and Interacting Hand Pose Estimation from a Single RGB Image. ECCV20. [PDF] [Project] [Code]
  Gyeongsik Moon, Shoou-i Yu, He Wen, Takaaki Shiratori, Kyoung Mu Lee

Transformer

• [HH.TR.8] MeMaHand: Exploiting Mesh-Mano Interaction for Single Image Two-Hand Reconstruction. CVPR23. [PDF]
  Congyi Wang, Feida Zhu, Shilei Wen

• [HH.TR.3] A2J-Transformer: Anchor-to-Joint Transformer Network for 3D Interacting Hand Pose Estimation from a Single RGB Image. CVPR23. [PDF] [Code]
  Changlong Jiang, Yang Xiao, Cunlin Wu, Mingyang Zhang, Jinghong Zheng, Zhiguo Cao, Joey Tianyi Zhou

• [HH.TR.2] 3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal. ECCV22. [PDF] [Project]
  Hao Meng, Sheng Jin, Wentao Liu, Chen Qian, Mengxiang Lin, Wanli Ouyang, Ping Luo

• [HH.TR.1] Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimation. CVPR22. [PDF] [Project]
  Shreyas Hampali, Sayan Deb Sarkar, Mahdi Rad, Vincent Lepetit

Graph

• [HH.GH.1] Interacting Attention Graph for Single Image Two-Hand Reconstruction. CVPR22. [PDF] [Code]
  Mengcheng Li, Liang An, Hongwen Zhang, Lianpeng Wu, Feng Chen, Tao Yu, Yebin Liu

Shape

• [HH.SP.1] Im2Hands: Learning Attentive Implicit Representation of Interacting Two-Hand Shapes. CVPR23. [PDF] [Project] [Code] Jihyun Lee, Minhyuk Sung, Honggyu Choi, Tae-Kyun Kim

Neural Rendering

• [HH.NR.1] HandNeRF: Neural Radiance Fields for Animatable Interacting Hands. CVPR23. [PDF] Zhiyang Guo, Wengang Zhou, Min Wang, Li Li, Houqiang Li

Gesture Prediction

• [HH.GP.2] Diverse 3D Hand Gesture Prediction from Body Dynamics by Bilateral Hand Disentanglement. CVPR23. [PDF]
  Xingqun Qi, Chen Liu, Muyi Sun, Lincheng Li, Changjie Fan, Xin Yu

• [HH.GP.1] Body2Hands: Learning to Infer 3D Hands from Conversational Gesture Body Dynamics. CVPR21. [PDF] [Project] [Code]
  Evonne Ng, Hanbyul Joo, Shiry Ginosar, Trevor Darrell

[back to top]

Full Body Reconstruction

Forward Kinematics

• [FBR.FK.7] PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images. arXiv22. [PDF] [Project]
  Hongwen Zhang, Yating Tian, Yuxiang Zhang, Mengcheng Li, Liang An, Zhenan Sun, Yebin Liu

• [FBR.FK.6] Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras. ICCV21. [PDF]
  Yuxiang Zhang, Zhe Li, Liang An, Mengcheng Li, Tao Yu, Yebin Liu

• [FBR.FK.5] Monocular Total Capture: Posing Face, Body, and Hands in the Wild. CVPR19. [PDF] [Project] [Code]
  Donglai Xiang, Hanbyul Joo, Yaser Sheikh

• [FBR.FK.4] Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies [PDF]
  Hanbyul Joo, Tomas Simon, Yaser Sheikh

• [FBR.FK.3] Expressive Body Capture: 3D Hands, Face, and Body from a Single Image. CVPR19. [PDF] [Project] [Code]
  Georgios Pavlakos, Vasileios Choutas*, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, Michael J. Black*

• [FBR.FK.2] FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integration. arXiv21. [PDF] [Code]
  Yu Rong, Takaaki Shiratori, Hanbyul Joo

• [FBR.FK.1] Monocular Expressive Body Regression through Body-Driven Attention. ECCV20. [PDF] [Project] [Code]
  Vasileios Choutas, Georgios Pavlakos, Timo Bolkart, Dimitrios Tzionas , Michael J. Black

Inverse Kinematics

• [FBR.IK.2] Pose2Pose: 3D Positional Pose-Guided 3D Rotational Pose Prediction for Expressive 3D Human Pose and Mesh Estimation. arXiv20. [PDF]
  Gyeongsik Moon, Kyoung Mu Lee

• [FBR.IK.1] Monocular Real-time Full Body Capture with Inter-part Correlations. CVPR21. [PDF]
  Yuxiao Zhou, Marc Habermann, Ikhsanul Habibie, Ayush Tewari, Christian Theobalt, Feng Xu

Shape

• [FBR.SP.7] ECON: Explicit Clothed humans Obtained from Normals. arXiv22. [PDF] [Project] [Code]
  Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, Michael J. Black

• [FBR.SP.6] SPAMs: Structured Implicit Parametric Models. CVPR22. [PDF] [Project]
  Pablo Palafox, Nikolaos Sarafianos, Tony Tung, Angela Dai

• [FBR.SP.5] Monocular Real-Time Volumetric Performance Capture. ECCV20. [PDF] [Code]
  Ruilong Li, Yuliang Xiu, Shunsuke Saito, Zeng Huang, Kyle Olszewski, Hao Li

• [FBR.SP.4] ARCH: Animatable Reconstruction of Clothed Humans. CVPR20. [PDF]
  Zeng Huang, Yuanlu Xu, Christoph Lassner, Hao Li, Tony Tung

• [FBR.SP.3] Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction. NeurIPS20. [PDF] [Code]
  Tong He, John Collomosse, Hailin Jin, Stefano Soatto

• [FBR.SP.2] PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization. CVPR20. [PDF] [Code]
  Shunsuke Saito, Tomas Simon, Jason Saragih, Hanbyul Joo

• [FBR.SP.1] PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization. CVPR19 [PDF] [Code]
  Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, Hao Li

Neural Rendering

• [FBR.NR.3] AvatarReX: Real-time Expressive Full-body Avatars. SIGGRAPH23. [PDF] [Project]
  Zerong Zheng, Xiaochen Zhao, Hongwen Zhang, Boning Liu, Yebin Liu

• [FBR.NR.2] X-Avatar: Expressive Human Avatars. CVPR23. [PDF] [Code] [Project]
  Kaiyue Shen, Chen Guo, Manuel Kaufmann, Juan Jose Zarate, Julien Valentin, Jie Song, Otmar Hilliges

• [FBR.NR.1] TotalSelfScan: Learning Full-body Avatars from Self-Portrait Videos of Faces, Hands, and Bodies. NeurIPS22. [PDF] [Project]
  Junting Dong, Qi Fang, Yudong Guo, Sida Peng, Qing Shuai, Xiaowei Zhou, Hujun Bao

Grasp

• [FBR.GR.3] Stochastic Whole-Body Grasping with Contact. arXiv21. [PDF] [Project] [Code]
  Yan Wu, Jiahao Wang, Yan Zhang, Siwei Zhang, Otmar Hilliges, Fisher Yu, Siyu Tang

• [FBR.GR.2] GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping. CVPR22. [PDF] [Project]
  Omid Taheri, Vasileios Choutas, Michael J. Black, Dimitrios Tzionas

• [FBR.GR.1] GRAB: A Dataset of Whole-Body Human Grasping of Objects. ECCV20. [PDF] [Project] [Code]
  Omid Taheri, Nima Ghorbani, Michael J. Black, Dimitrios Tzionas

In-Sence

• [FBR.IS.6] Generating Continual Human Motion in Diverse 3D Scenes. arXiv23. [PDF]
  Aymen Mir, Xavier Puig, Angjoo Kanazawa, Gerard Pons-Moll

• [FBR.IS.5] Stochastic Scene-Aware Motion Prediction. ICCV21. [PDF] [Project] [Code]
  Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, Michael Black

• [FBR.IS.4] PLACE: Proximity Learning of Articulation and Contact in 3D Environments. 3DV20. [PDF] [Project] [Code]
  Siwei Zhang, Yan Zhang, Qianli Ma, Michael J. Black, Siyu Tang

• [FBR.IS.3] Populating 3D Scenes by Learning Human-Scene Interaction. CVPR21. [PDF] [Project] [Code]
  Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, Michael J. Black

• [FBR.IS.2] The Wanderings of Odysseus in 3D Scenes. CVPR22. [PDF] [Project]
  Yan Zhang Siyu Tang

• [FBR.IS.1] Generating 3D People in Scenes without People. CVPR20. [PDF] [Code]
  Yan Zhang, Mohamed Hassan, Heiko Neumann, Michael J. Black, Siyu Tang

[back to top]

Tools

Model

• [T.MD.6] NIMBLE
• [T.MD.5] HTML
• [T.MD.4] mano
• [T.MD.3] manotorch
• [T.MD.2] manopth
• [T.MD.1] spheremesh

MoCap

• [T.MC.7] Mediapipe-TFLite
• [T.MC.6] AlphaPose
• [T.MC.5] EasyMocap
• [T.MC.4] MediaPipe
• [T.MC.3] frankmocap
• [T.MC.2] HandMotionViewer
• [T.MC.1] openpose

Data

• [T.DA.1] HandData

[back to top]