Human Motion Capture

Collecting papers about human motion capture

⭐🔥💪 I will update each day and add more details about every paper ☀️☀️☀️

NOTE : Since motion capture is a field that has been studied for many years, I will collect the up to date papers first and then try to gather the old but classical one.

methods based directly on video

1. Physically Plausible Monocular 3D Motion Capture [project]
2. "Neural PhysCap" Neural Monocular 3D Human Motion Capture with Physical Awareness [project]
3. XNect: real-time multi-person 3D motion capture with a single RGB camera[project]
4. 4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras [code]
5. VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera [project]
6. TesseTrack: End-to-End Learnable Multi-Person Articulated 3D Pose Tracking [project]
7. Synergetic Reconstruction from 2D Pose and 3D Motion for Wide-Space Multi-Person Video Motion Capture in the Wild [project]
8. MonoPerfCap: Human Performance Capture from Monocular Video [project]
9. Capturing Detailed Deformations of Moving Human Bodies [project]
10. Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras [project]
11. Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos [pdf]
12. Direct Multi-view Multi-person 3D Human Pose Estimation [code]
13. Human Performance Capture from Monocular Video in the Wild [pdf] | [code]
14. Camera Distortion-aware 3D Human Pose Estimation in Video with Optimization-based Meta-Learning[pdf] | [code]
15. Human Dynamics from Monocular Video with Dynamic Camera Movements [code] | [project]
16. AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation [code]
17. Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capture [project]
18. Permutation-Invariant Relational Network for Multi-person 3D Pose Estimation [pdf]
19. PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision [code]
20. Generalizable Human Pose Triangulation [code]
21. Differentiable Dynamics for Articulated 3d Human Motion Reconstruction [pdf]
22. VTP: Volumetric Transformer for Multi-view Multi-person 3D Pose Estimation [pdf]
23. Trajectory Optimization for Physics-Based Reconstruction of 3d Human Pose from Monocular Video [pdf]
24. Learning Variational Motion Prior for Video-based Motion Capture [pdf]
25. Accurate and Efficient Absolute 3D Human Pose Estimation Trained on Dozens of Datasets [project]
26. GFPose: Learning 3D Human Pose Prior with Gradient Fields [[project]
27. DiffPose: Toward More Reliable 3D Pose Estimation [pdf] | [project]
28. Learning 3D Human Pose Estimation From Dozens of Datasets Using a Geometry-Aware Autoencoder To Bridge Between Skeleton Formats [project]
29. Scene-Aware 3D Multi-Human Motion Capture from a Single Camera [project]
30. Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition [project]
31. 3D Human Pose Estimation via Intuitive Physics [project]
32. GraMMaR: Ground-aware Motion Model for 3D Human Motion Reconstruction [project]
33. Real-time Monocular Full-body Capture in World Space via Sequential Proxy-to-Motion Learning [project]
34. Back to Optimization: Diffusion-based Zero-Shot 3D Human Pose Estimation [pdf]

methods based on body model such as smpl/smpl-x

Note that there are some papers that I don't list here most because I have tested it and the result is not so good(e,g. frankmocap,VIBE and so on)

1. Contact and Human Dynamics from Monocular Video [code]
2. DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras [project]
3. Monocular, One-stage, Regression of Multiple 3D People [code]
4. BEV:Putting People in their Place,Monocular Regression of 3D People in Depth [project]
5. PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop [project]
6. End-to-End Human Pose and Mesh Reconstruction with Transformers [code]
7. Pose2Pose: 3D Positional Pose-Guided 3D Rotational Pose Prediction for Expressive 3D Human Pose and Mesh Estimation[paper] | [code]
8. AGORA: Avatars in Geography Optimized for Regression Analysis [code]
9. Real-time RGBD-based Extended Body Pose Estimation [code]
10. EasyMocap [code]
11. PanoMan: Sparse Localized Components–based Model for Full Human Motions [paper]
12. Monocular Total Capture: Posing Face, Body and Hands in the Wild [code]
13. Full-body motion capture for multiple closely interacting persons [paper]
14. PARE: Part Attention Regressor for 3D Human Body Estimation [project] | [code]
15. DSFN: Dynamic Surface Function Networks for Clothed Human Bodies [code]
16. 3D Human Pose and Shape Estimation Through Collaborative Learning and Multi-view Model-fitting [code]
17. On Self-Contact and Human Pose [project]
18. KAMA: 3D Keypoint Aware Body Mesh Articulation [paper] | [video]
19. Bilevel Online Adaptation for Out-of-Domain Human Mesh Reconstruction [code]
20. Multi-person Implicit Reconstruction from a Single Image [paper] | [video]
21. Body Meshes as Points [code]
22. Collaborative Regression of Expressive Bodies using Moderation [project]
23. Deep3DPose: Realtime Reconstruction of Arbitrarily Posed Human Bodies from Single RGB Images [paper]
24. RobustFusion: Robust Volumetric Performance Reconstruction under Human-object Interactions from Monocular RGBD Stream [paper]
25. Monocular Real-time Full Body Capture with Inter-part Correlations [project]
26. Learning Complex 3D Human Self-Contact [pdf]
27. Learning Local Recurrent Models for Human Mesh Recovery [pdf]
28. LASOR: Learning Accurate 3D Human Pose and Shape Via Synthetic Occlusion-Aware Data and Neural Mesh Rendering [pdf]
29. HuMoR: 3D Human Motion Model for Robust Pose Estimation [code]
30. Probabilistic Modeling for Human Mesh Recovery [code]
31. NeMF: Neural Motion Fields for Kinematic Animation [pdf]
32. Encoder-decoder with Multi-level Attention for 3D Human Shape and PoseEstimation [code]
33. LEMO: Learning Motion Priors for 4D Human Body Capture in 3D Scenes [ptoject]
34. Shape-aware Multi-Person Pose Estimation from Multi-View Images [project]
35. SPEC: Seeing People in the Wild with an Estimated Camera [project]
36. Learning to Regress Bodies from Images using Differentiable Semantic Rendering [code]
37. Mesh Graphormer [code]
38. VoteHMR: Occlusion-Aware Voting Network for Robust 3D Human Mesh Recovery from Partial Point Clouds [pdf] | [code]
39. Leveraging MoCap Data for Human Mesh Recovery [pdf]
40. PIXIE: Collaborative Regression of Expressive Bodies [code]
41. Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Cameras [pdf] }| [code]
42. Hierarchical Kinematic Probability Distributions for 3D Human Shape and Pose Estimation from Images in the Wild [code]
43. Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation [code]
44. Tracking People with 3D Representations [code]
45. LatentHuman: Shape-and-Pose Disentangled Latent Representation for for Human Bodies[project]
46. PoseBERT [project]
47. Camera Motion Agnostic 3D Human Pose Estimation [code]
48. GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras [project]
49. Revitalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation [project]
50. ICON: Implicit Clothed humans Obtained from Normals [code]
51. Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation from Monocular Video [project]
52. Occluded Human Mesh Recovery [pdf]
53. Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation [code]
54. Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimation [code]
55. CHOMP: Occluded Human Body Capture with Self-Supervised Spatial-Temporal Motion Prior [code]
56. PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images [project]
57. Benchmarking 3D Pose and Shape Estimation Beyond Algorithms [code]
58. Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers [code]
59. CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation [code]
60. Learning Visibility for Robust Dense Human Body Estimation [code]
61. XRMocap - multi-view motion capture [code]
62. CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation [code]
63. PLIKS: A Pseudo-Linear Inverse Kinematic Solver for 3D Human Body Estimation [pdf]
64. FuRPE: Learning Full-body Reconstruction from Part Experts [code]
65. NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action [project]
66. IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation [project]
67. Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers [project]
68. Decoupling Human and Camera Motion from Videos in the Wild [project]
69. Capturing the motion of every joint: 3D human pose and shape estimation with independent tokens [project]
70. PoseExaminer: Automated Testing of Out-of-Distribution Robustness in Human Pose and Shape Estimation [code]
71. GATOR: Graph-Aware Transformer with Offset-Disentangled Regression for Human Mesh Reconstruction from a 2D Pose [project]
72. BoPR: Body-aware Part Regressor for Human Shape and Pose Estimation [pdf]
73. Contact, Human and Object REconstruction from a single RGB image [code]
74. One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer [project]
75. EgoHMR: Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Views [project]
76. HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recovery [project]
77. Learning Analytical Posterior Probability for Human Mesh Recovery [project]
78. Sampling is Matter: Point-guided 3D Human Mesh Reconstruction [code]
79. SHOW: Synchronous HOlistic body in the Wild [code]
80. HuManiFlow: Ancestor-Conditioned Normalising Flows on SO(3) Manifolds for Human Pose and Shape Distribution Estimation [code]
81. NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation [code]
82. XFormer: Fast and Accurate Monocular 3D Body Capture [pdf]
83. IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation [code]
84. Learning Analytical Posterior Probability for Human Mesh Recovery [code]
85. Humans in 4D: Reconstructing and Tracking Humans with Transformers [code]
86. NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D HPS [code]
87. Implicit 3D Human Mesh Recovery using Consistency with Pose and Shape from Unseen-view [pdf]
88. BEDLAM: Bodies Exhibiting Detailed Lifelike Animated Motion [project]
89. MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling [code]
90. Tracking People by Predicting 3D Appearance, Location & Pose [code]
91. Learning 3D Human Shape and Pose from Dense Body Parts [code]
92. Body Knowledge and Uncertainty Modeling for Monocular 3D Human Body Reconstruction [pdf]
93. JOTR: 3D Joint Contrastive Learning with Transformers for Occluded Human Mesh Recovery [code]
94. Semantify:Simplifying the Control of 3D Morphable Models using CLIP [code]

human 3d pose estimation

1. Freemocap [code]
2. MeTRAbs: Metric-Scale Truncation-Robust Heatmaps for Absolute 3D Human Pose Estimation [code]
3. Part-aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking [code]
4. Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo [code]
5. 3D Multi-Person Pose Estimation by Integrating Top-Down and Bottom-Up Networks [code]
6. CanonPose: Self-Supervised Monocular 3D Human Pose Estimation in the Wild [paper]
7. 3D Human Pose Estimation with Spatial and Temporal Transformers [code]
8. Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild [paper]
9. AdaFuse: Adaptive Multiview Fusion for Accurate Human Pose Estimation in the Wild [code]
10. Improving Robustness and Accuracy via Relative Information Encoding in 3D Human Pose Estimation [code]
11. Probabilistic-Monocular-3D-Human-Pose-Estimation-with-Normalizing-Flows [code]
12. VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wild [pdf]
13. MetaPose: Fast 3D Pose from Multiple Views without 3D Supervision [pdf]
14. CrossFormer: Cross Spatio-Temporal Transformer for 3D Human Pose Estimation[pdf]
15. Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation [project]
16. Ray3D: ray-based 3D human pose estimation for monocular absolute 3D localization [code]
17. PedRecNet: Multi-task deep neural network for full 3D human pose and orientation estimation [code]
18. Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation [code]
19. PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision [code]
20. A Dual-Masked Auto-Encoder for Robust Motion Capture with Spatial-Temporal Skeletal Token Completion [pdf]
21. DeciWatch: A Simple Baseline for 10× Efficient 2D and 3D Pose Estimation [project]
22. VirtualPose: Learning Generalizable 3D Human Pose Models from Virtual Data [code]
23. Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projection [pdf]
24. KinePose: A temporally optimized inverse kinematics technique for 6DOF human pose estimation with biomechanical constraint [code]
25. A Dual-Masked Auto-Encoder for Robust Motion Capture with Spatial-Temporal Skeletal Token Completion [code]
26. HDFormer: High-order Directed Transformer for 3D Human Pose Estimation [pdf]
27. HTNet: Human Topology Aware Network for 3D Human Pose Estimation [project]
28. Proactive Multi-Camera Collaboration for 3D Human Pose Estimation [project]
29. GFPose: Learning 3D Human Pose Prior with Gradient Fields [project]
30. Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregation [code]
31. Global Adaptation meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimation [pdf]
32. PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation [project]
33. PoseRAC: Pose Saliency Transformer for Repetitive Action Counting [code]
34. CrossFormer: Cross Spatio-Temporal Transformer for 3D Human Pose Estimation [pdf]
35. Motion-DVAE: Unsupervised learning for fast human motion denoising [project]
36. Iterative Graph Filtering Network for 3D Human Pose Estimation [code]

simplify optical or inertial based motion capture

1. Real-Time Multi-person Motion Capture from Multi-view Video and IMUs[paper]
2. TransPose [code]
3. Physical Inertial Poser (PIP):Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensors [project]
4. EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers [code]
5. HybridCap: Inertia-aid Monocular Capture of Challenging Human Motions [pdf]
6. Transformer Inertial Poser: Attention-based Real-time Human Motion Reconstruction from Sparse IMUs [project]
7. DeMoCap: Low-cost Marker-based Motion Capture [code]
8. LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors [pdf]
9. AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensing [code]
10. HybridTrak: Adding Full-Body Tracking to VR Using an Off-the-Shelf Webcam [video]
11. FusePose: IMU-Vision Sensor Fusion in Kinematic Space for Parametric Human Pose Estimation [pdf]
12. Fusion Poser: 3D Human Pose Estimation using Sparse IMUs and Head Tracker in real-time [code]
13. QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars [pdf]
14. TIP: Real-time Human Motion Reconstruction from Sparse IMUs with Simultaneous Terrain Generation [pdf]
15. FLAG: Flow-based 3D Avatar Generation from Sparse Observations [project]
16. Combining Motion Matching and Orientation Prediction to Animate Avatars for Consumer-Grade VR Devices [project]
17. Faster Deep Inertial Pose Estimation with Six Inertial Sensors [code]
18. Neural3Points: Learning to Generate Physically Realistic Full-body Motion for Virtual Reality Users [pdf]

human motion capture in 3d scene

1. 4D Human Body Capture from Egocentric Video via 3D Scene Grounding [project]
2. LEMO Learning Motion Priors for 4D Human Body Capture in 3D Scenes [project]
3. Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors [project]
4. EgoBody Dataset:Human Body Shape, Motion and Social Interactions from Head-Mounted Devices[project]
5. GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping [project]
6. HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDAR [project]
7. Human-Aware Object Placement for Visual Environment Reconstruction [project]
8. COAP: Compositional Articulated Occupancy of People [project]
9. BEHAVE: Dataset and Method for Tracking Human Object Interactions [project]
10. Synthesizing Long-Term 3D Human Motion and Interaction in 3D [project]
11. Visually plausible human-object interaction capture from wearable sensors [project]
12. HULC: 3D HUman Motion Capture with Pose Manifold Sampling and Dense Contact Guidance [project]
13. Towards Diverse and Natural Scene-aware 3D Human Motion Synthesis [pdf]
14. MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenes [project]
15. The One Where They Reconstructed 3D Humans and Environments in TV Shows [project]
16. Stochastic Scene-Aware Motion Prediction [code]
17. COINS : Compositional Human-Scene Interaction Synthesis with Semantic Control [projects]
18. Placing Human Animations into 3D Scenes by Learning Interaction- and Geometry-Driven Keyframes [[project]
19. InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interaction [project]
20. HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes [project]
21. SLOPER4D: A Scene-Aware Dataset for Global 4D Human Pose Estimation in Urban Environments [code]
22. EgoLocate:Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensors [project]
23. Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesis [project]
24. Learning Human Mesh Recovery in 3D Scenes [project]
25. MIME: Human-Aware 3D Scene Generation [project]
26. TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environments [project]
27. CIMI4D: A Large Multimodal Climbing Motion Dataset under Human-scene Interactions [project]
28. IMoS: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions [code]