← ArchivePaper2016
An Anatomically Constrained Local Deformation Model for Monocular Face Capture
Anatomically constrained local deformation model for monocular face capture that prevents physically implausible face reconstructions.
Abstract
We present a new anatomically-constrained local face model and fitting approach for tracking 3D faces from 2D motion data in very high quality. In contrast to traditional global face models, often built from a large set of blendshapes, we propose a local deformation model composed of many small subspaces spatially distributed over the face. Our local model offers far more flexibility and expressiveness than global blendshape models, even with a much smaller model size. This flexibility would typically come at the cost of reduced robustness, in particular during the under-constrained task of monocular reconstruction. However, a key contribution of this work is that we consider the face anatomy and introduce subspace skin thickness constraints into our model, which constrain the face to only valid expressions and helps counteract depth ambiguities in monocular tracking. Given our new model, we present a novel fitting optimization that allows 3D facial performance reconstruction from a single view at extremely high quality, far beyond previous fitting approaches. Our model is flexible, and can be applied also when only sparse motion data is available, for example with marker-based motion capture or even face posing from artistic sketches.
How to read this
- Category
- Method: an anatomically constrained local face model for monocular capture
- Contributions
- Proposes a local face deformation model made of many small spatially distributed subspaces rather than a single global blendshape basis, giving more flexibility at smaller model size
- Introduces subspace skin-thickness (anatomy) constraints that restrict the face to valid expressions and counteract monocular depth ambiguity
- Presents a fitting optimization for high-quality single-view 3D facial performance reconstruction, applicable also to sparse marker-based motion data
- Context
- Builds on production facial-capture systems such as Medusa (beeler-medusa-2012), trading global blendshape models for a local, anatomy-constrained alternative.Builds on: High-Quality Passive Facial Performance Capture Using Anchor Frames
- Correctness
- The key assumption is that skin-thickness anatomical priors keep the flexible local model from overfitting implausible shapes under the under-constrained monocular setting; demonstrated on high-quality tracking, but robustness still hinges on the validity of those anatomical constraints and 2D motion data quality.
- Clarity
- Moderately technical; a first pass conveys the local-subspace-plus-anatomy idea, but the fitting optimization needs a second pass.
- How to read it
- Read first for why local subspaces plus skin-thickness constraints beat global blendshapes for monocular capture; do a second pass on the fitting optimization if you intend to implement or compare against it.
Related work
- High Resolution Passive Facial Performance Capture 2010 / SIGGRAPH
- High Fidelity Facial Animation Capture and Retargeting with Contours 2013 / SCA
- Rigid Stabilization of Facial Expressions 2011 / SIGGRAPH
- Driving High-Resolution Facial Scans with Video Performance Capture 2015 / SIGGRAPH
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →