← ArchivePaper2014
Facial Performance Enhancement Using Dynamic Shape Space Analysis
Amit H. Bermano, Derek Bradley, Thabo Beeler, Fabio Zund, Derek Nowrouzezahrai, Ilya Baran, Olga Sorkine-Hornung, Hanspeter Pfister, Markus Gross
Enhances dynamic facial performance captures by propagating high-frequency detail from a reference scan through the sequence.
Abstract
The facial performance of an individual is inherently rich in subtle deformation and timing details. Although these subtleties make the performance realistic and compelling, they often elude both motion capture and hand animation. We present a technique for adding fine-scale details and expressiveness to low-resolution art-directed facial performances, such as those created manually using a rig, via marker-based capture, by fitting a morphable model to a video, or through Kinect reconstruction using recent faceshift technology. We employ a high-resolution facial performance capture system to acquire a representative performance of an individual in which he or she explores the full range of facial expressiveness. From the captured data, our system extracts an expressiveness model that encodes subtle spatial and temporal deformation details specific to that particular individual. Once this model has been built, these details can be transferred to low-resolution art-directed performances. We demonstrate results on various forms of input; after our enhancement, the resulting animations exhibit the same nuances and fine spatial details as the captured performance, with optional temporal enhancement to match the dynamics of the actor. Finally, we show that our technique outperforms the current state-of-the-art in example-based facial animation.
How to read this
- Category
- Method: facial performance detail enhancement / transfer
- Contributions
- A technique to add fine-scale spatial detail and expressiveness to low-resolution, art-directed facial performances
- An expressiveness model extracted from a high-resolution capture of one individual exploring their full expressive range, encoding subtle spatial and temporal deformation details
- Transfer of those details onto inputs from rigs, marker capture, morphable-model video fits, or Kinect reconstruction
- Context
- Relates to high-quality facial capture (Beeler et al.'s 'High-Quality Passive Facial Performance Capture Using Anchor Frames') and uses it as the source of the per-individual detail model that is then propagated to coarse performances.Builds on: High-Quality Passive Facial Performance Capture Using Anchor Frames
- Correctness
- The enhancement is person-specific: it assumes a representative high-resolution capture of the same individual exists and that the low-res input is expressively compatible; results are demonstrated across several input types, but transfer quality is bounded by how well the captured range covers the target performance.
- Clarity
- Accessible in concept; a first pass conveys the capture-then-transfer idea, a second pass pays off for the shape-space analysis and the spatial/temporal detail formulation.
- How to read it
- Read for the pipeline (build expressiveness model, then transfer); focus on what the dynamic shape space encodes and the assumed match between reference capture and target, then a second pass on the math if you intend to reproduce it.
Built upon by
Nothing yet.
Related work
- Realtime Facial Animation with On-the-fly Correctives 2013 / SIGGRAPH
- The Digital Emily Project: Achieving a Photorealistic Digital Actor 2010 / Tech Note
- Online Modeling for Realtime Facial Animation 2013 / SIGGRAPH
- Animatomy: An Animator-Centric, Anatomically Inspired System for 3D Facial Modeling, Animation and Transfer 2022 / SIGGRAPH Asia
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →