← ArchivePaper2010

High Resolution Passive Facial Performance Capture

Derek Bradley, Wolfgang Heidrich, Tiberiu Popa, Alla Sheffer

SIGGRAPHAcademic1 descendantFacial

Multi-view passive capture system for facial performance without markers, recovering high-resolution dynamic geometry from synchronized cameras.

Open the paper ↗Trace in the lineage graph

Abstract

We introduce a purely passive facial capture approach that uses only an array of video cameras, but requires no template facial geometry, no special makeup or markers, and no active lighting. We obtain initial geometry using multi-view stereo, and then use a novel approach for automatically tracking texture detail across the frames. As a result, we obtain a high-resolution sequence of compatibly triangulated and parameterized meshes. The resulting sequence can be rendered with dynamically captured textures, while also consistently applying texture changes such as virtual makeup.

How to read this

Category: Capture system: passive multi-view facial performance capture
Contributions: A purely passive facial capture approach using only an array of video cameras, with no template geometry, no markers or makeup, and no active lighting
Initial geometry from multi-view stereo plus a method for automatically tracking texture detail across frames
Output of a high-resolution sequence of compatibly triangulated and parameterized meshes that can be rendered with captured textures and support texture edits such as virtual makeup
Context: A markerless, passive alternative in the performance-driven facial animation lineage (Williams, Performance-Driven Facial Animation), relying on multi-view stereo plus temporal texture tracking.Builds on: Performance-Driven Facial Animation
Correctness: Relies on sufficient skin texture and synchronized multi-view coverage for stereo and tracking; readers should keep in mind that without markers or active lighting, tracking robustness depends on visible texture detail and the capture-studio camera setup.
Clarity: Accessible; a first pass conveys the passive, template-free pipeline, do a second pass for the texture-tracking method that yields temporally consistent meshes.
How to read it: Focus on what makes it template- and marker-free and on the cross-frame texture tracking; a second pass pays off on the tracking and mesh-correspondence details if you compare it to active-lighting systems.

Builds on

Performance-Driven Facial Animation 1990

Built upon by

High Fidelity Facial Animation Capture and Retargeting with Contours 2013

Related work

Performance-Driven Facial Animation 1990 / SIGGRAPH
High-Quality Passive Facial Performance Capture Using Anchor Frames 2011 / SIGGRAPH
Vision-based control of 3D facial animation 2003 / SCA
High-Quality Face Capture Using Anatomical Muscles 2019 / CVPR

Keywords

multi-view stereo optical flow surface tracking markerless capture facial performance capture temporal drift correction texture mapping passive facial capture motion capture deformable geometry stereo reconstruction

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →