← ArchivePaper2025
BioPose: Biomechanically-accurate 3D Pose Estimation from Monocular Videos
Farnoosh Koleini, Muhammad Usama Saleem, Pu Wang, Hongfei Xue, Ahmed Helmy, Abbey Fenwick
Three-stage pipeline combining multi-query mesh recovery, neural inverse kinematics, and 2D-informed refinement to produce biomechanically plausible human poses from monocular video.
Abstract
Recent advancements in 3D human pose estimation from single-camera images and videos have relied on parametric models, like SMPL. However, these models over-simplify anatomical structures, limiting their accuracy in capturing true joint locations and movements, which reduces their applicability in biomechanics, healthcare, and robotics. Biomechanically accurate pose estimation, on the other hand, typically requires costly marker-based motion capture systems and optimization techniques in specialized labs. To bridge this gap, we propose BioPose, a novel learning-based framework for predicting biomechanically accurate 3D human pose directly from monocular videos. BioPose includes three key components: a Multi-Query Human Mesh Recovery model (MQ-HMR), a Neural Inverse Kinematics (NeurIK) model, and a 2D-informed pose refinement technique. MQ-HMR leverages a multi-query deformable transformer to extract multi-scale fine-grained image features, enabling precise human mesh recovery. NeurIK treats the mesh vertices as virtual markers, applying a spatial-temporal network to regress biomechanically accurate 3D poses under anatomical constraints. To further improve 3D pose estimations, a 2D-informed refinement step optimizes the query tokens during inference by aligning the 3D structure with 2D pose observations.
How to read this
- Category
- Method: biomechanical 3D pose estimation
- Contributions
- BioPose, a learning-based framework predicting biomechanically accurate 3D human pose directly from monocular video
- Combines a Multi-Query Human Mesh Recovery model (MQ-HMR) using a multi-query deformable transformer, a Neural Inverse Kinematics (NeurIK) model treating mesh vertices as virtual markers, and a 2D-informed pose refinement step
- Aims to recover anatomically constrained joint locations without marker-based motion capture
- Context
- Responds to the anatomical over-simplification of parametric models like SMPL and shares the biomechanical-accuracy goal of From Skin to Skeleton, but targets estimation from a single camera rather than capture rigs.Builds on: From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans
- Correctness
- The premise is that mesh vertices can act as virtual markers feeding a neural IK under anatomical constraints to approximate marker-based accuracy from monocular video; readers should be cautious about how close to true marker-based ground truth this gets and on which datasets and motion types it was validated.
- Clarity
- Pipeline is clearly staged (three components), so a first pass conveys the architecture; the deformable-transformer and NeurIK details need a second pass.
- How to read it
- First pass for the three-stage pipeline and the virtual-markers idea; second pass on NeurIK and the anatomical constraints if you care about biomechanical fidelity, and check the evaluation protocol against marker-based references before trusting accuracy claims.
Built upon by
Nothing yet.
Related work
- SFV: Reinforcement Learning of Physical Skills from Video 2018 / SIGGRAPH Asia
- AddBiomechanics: Automating model scaling, inverse kinematics, and inverse dynamics from human motion data through sequential optimization 2023 / PLOS ONE
- From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans 2023 / SIGGRAPH Asia
- Reconstructing Humans with a Biomechanically Accurate Skeleton 2025 / CVPR
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →