← ArchivePaper1999
A Morphable Model for the Synthesis of 3D Faces
The 3D morphable face model: a statistical face space fit to images, ancestor of every data-driven face model since.
Abstract
This paper introduces a technique for modeling textured 3D faces from a dataset of prototypical laser scans, deriving a morphable face model by transforming the shape and texture of example faces into a vector space representation. New faces and expressions are generated as linear combinations of the prototypes, with shape and texture constraints derived from the statistics of the example faces regulating the naturalness of results. An analysis-by-synthesis matching algorithm reconstructs 3D shape and texture from one or more photographs by optimizing model coefficients and rendering parameters, and a bootstrapping optic flow procedure establishes dense one-to-one correspondence across the example faces. The system also enables manipulation of complex facial attributes such as gender, fullness, and distinctiveness.
How to read this
- Category
- Method: a statistical (morphable) 3D face model
- Contributions
- A morphable face model: shape and texture of laser-scanned prototype faces cast into a vector space, with new faces as regulated linear combinations
- An analysis-by-synthesis algorithm that reconstructs 3D shape and texture from one or more photographs by optimizing model and rendering parameters
- A bootstrapping optic-flow procedure for dense correspondence, plus control of attributes like gender, fullness, and distinctiveness
- Context
- A foundational statistical-shape-model approach for faces, ancestor of essentially all later data-driven 3D face models, drawing on PCA-style example-based modeling rather than a single prior graphics paper.
- Correctness
- The face space is only as expressive as its laser-scan dataset and the assumption of dense correspondence and linear shape/texture combination; analysis-by-synthesis fitting is an optimization that can depend on initialization and imaging conditions.
- Clarity
- Dense, with statistics and optimization; a first pass conveys the model concept, a careful second (and third) pass is needed for the correspondence and fitting machinery.
- How to read it
- First pass for the linear face-space idea and analysis-by-synthesis framing; second/third pass on the correspondence bootstrapping and the fitting optimization if you work with face models.
Builds on
Nothing in the archive, this is a starting point.
Built upon by
- Face Transfer with Multilinear Models 2005
- The Digital Emily Project: Achieving a Photorealistic Digital Actor 2010
- 3D Shape Regression for Real-Time Facial Animation 2013
- FaceWarehouse: A 3D Facial Expression Database for Visual Computing 2014
- Vdub: Modifying Face Video of Actors for Plausible Visual Alignment to a Dubbed Audio Track 2015
- Face2Face: Real-Time Face Capture and Reenactment of RGB Videos 2016
- Reconstruction of Personalized 3D Face Rigs from Monocular Video 2016
- Learning a Model of Facial Shape and Expression from 4D Scans 2017
- 3D Morphable Face Models: Past, Present and Future 2021
- i3DMM: Deep Implicit 3D Morphable Model of Human Heads 2021
Related work
- 3D Morphable Face Models: Past, Present and Future 2021 / SIGGRAPH
- Practice and Theory of Blendshape Facial Models 2014 / Eurographics
- Learning an Animatable Detailed 3D Face Model from In-The-Wild Images 2021 / SIGGRAPH
- FaceWarehouse: A 3D Facial Expression Database for Visual Computing 2014 / TVCG
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →