← ArchivePaper2019
Expressive Body Capture: 3D Hands, Face, and Body from a Single Image
Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. Osman, Dimitrios Tzionas, Michael J. Black
SMPL-X unifies body, face, and fully articulated hands into a single 10,475-vertex parametric model with 54 joints and a learned variational pose prior.
Abstract
To facilitate the analysis of human actions, interactions and emotions, we compute a 3D model of human body pose, hand pose, and facial expression from a single monocular image. To achieve this, we use thousands of 3D scans to train a new, unified, 3D model of the human body, SMPL-X, that extends SMPL with fully articulated hands and an expressive face. Learning to regress the parameters of SMPL-X directly from images is challenging without paired images and 3D ground truth. Consequently, we follow the approach of SMPLify, which estimates 2D features and then optimizes model parameters to fit the features. We improve on SMPLify in several significant ways: (1) we detect 2D features corresponding to the face, hands, and feet and fit the full SMPL-X model to these; (2) we train a new neural network pose prior using a large MoCap dataset; (3) we define a new interpenetration penalty that is both fast and accurate; (4) we automatically detect gender and the appropriate body models (male, female, or neutral); (5) our PyTorch implementation achieves a speedup of more than 8x over Chumpy. We use the new method, SMPLify-X, to fit SMPL-X to both controlled images and images in the wild. We evaluate 3D accuracy on a new curated dataset comprising 100 images with pseudo ground-truth. This is a step towards automatic expressive human capture from monocular RGB data.
How to read this
- Category
- Method + model: a unified parametric body model and single-image fitting pipeline
- Contributions
- SMPL-X, a unified parametric model extending the body with fully articulated hands and an expressive face
- SMPLify-X, which fits SMPL-X to detected 2D face, hand and foot features from a single monocular image
- Supporting pieces: a neural-network pose prior trained on MoCap, a fast interpenetration penalty, automatic gender detection, and a faster PyTorch implementation
- Context
- Extends SMPL (Loper et al.) toward whole-body expressiveness, drawing the face component from the FLAME line (Li et al.) and relating to single-image pose estimation work such as VNect (Mehta et al.).Builds on: SMPL: A Skinned Multi-Person Linear Model · Learning a Model of Facial Shape and Expression from 4D Scans · VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera at over 30fps
- Correctness
- An optimization-based fit to 2D features, so results depend on detector quality and the learned pose prior; demonstrated on controlled and in-the-wild images, but single-image monocular fitting carries inherent depth and occlusion ambiguity that a reader should keep in mind.
- Clarity
- Readable structure; a first pass conveys the model and pipeline, while the energy terms and pose prior reward a second pass.
- How to read it
- First pass for what SMPL-X adds over SMPL and how SMPLify-X differs from SMPLify; do a second pass on the objective terms (pose prior, interpenetration penalty) if you intend to fit or extend the model.
Builds on
Built upon by
Related work
- SUPR: A Sparse Unified Part-Based Human Representation 2022 / Eurographics
- SMPL: A Skinned Multi-Person Linear Model 2015 / SIGGRAPH Asia
- ATLAS: Decoupling Skeletal and Shape Parameters for Expressive Parametric Human Modeling 2025 / ICCV
- NIMBLE: A Non-rigid Hand Model with Bones and Muscles 2022 / TOG
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →