← ArchivePaper2020

Data-driven Extraction and Composition of Secondary Dynamics in Facial Performance Capture

Gaspard Zoss, Eftychios Sifakis, Markus Gross, Derek Bradley, Thabo Beeler

SIGGRAPHDisney Research3 citesFacial

Data-driven method to extract and separately compose secondary dynamic effects from facial performance capture for enhanced realism.

Open the paper ↗Trace in the lineage graph

Abstract

Performance capture of expressive subjects, particularly facial performances acquired with high spatial resolution, will inevitably incorporate some fraction of motion that is due to inertial effects and dynamic overshoot due to ballistic motion. This is true in most natural capture environments where the actor is able to move freely during their performance, rather than being tethered to a fixed position. Normally these secondary dynamic effects are unwanted, as the captured facial performance is often retargeted to different head motion, and sometimes to completely different characters, and in both cases the captured dynamic effects should be removed and new secondary effects should be added. This paper advances the hypothesis that for a highly constrained elastic medium such as the human face, these secondary inertial effects are predominantly due to the motion of the underlying bony structures (cranium and mandible). Our work aims to compute and characterize the difference between the captured dynamic facial performance, and a speculative quasistatic variant of the same motion should the inertial effects have been absent.

How to read this

Category: Method: data-driven analysis of facial performance capture
Contributions: Extracts unwanted secondary inertial/overshoot dynamics from captured facial performances so they can be removed before retargeting
Advances the hypothesis that facial secondary inertia is predominantly driven by motion of the cranium and mandible
Computes the difference between the captured dynamic performance and a speculative quasistatic variant, enabling separate composition of new secondary effects
Context: Builds on anatomically grounded facial rigging, related to Zoss et al.'s 'An Empirical Rig for Jaw Animation', extending it from static jaw structure to dynamic inertial effects.Builds on: An Empirical Rig for Jaw Animation
Correctness: Central assumption is that the face is a constrained elastic medium whose secondary motion is dominated by bony-structure (cranium/mandible) motion; validated on captured performances, but the bone-driven hypothesis may under-model soft-tissue-driven dynamics and depends on accurate skull/jaw tracking.
Clarity: Specialized; a first pass conveys the extract-and-recompose hypothesis, a second pass is needed for the quasistatic-difference formulation.
How to read it: Focus on the cranium/mandible hypothesis and the quasistatic-versus-captured decomposition; second pass if you work on retargeting or dynamic facial realism.

Builds on

An Empirical Rig for Jaw Animation 2018

Built upon by

Nothing yet.

Related work

Semi-Supervised Video-Driven Facial Animation Transfer for Production 2021 / SIGGRAPH Asia
BlendForces: A Dynamic Framework for Facial Animation 2016 / CGF
FaceLab: Scalable Facial Performance Capture for Visual Effects 2020 / DigiPro
FaceBaker: Baking Character Facial Rigs with Machine Learning 2020 / SIGGRAPH

Keywords

secondary dynamics facial performance capture soft-tissue motion quasistatic performance dynamics removal dynamics synthesis neural network skull motion kinematic history surface stretch inertial deformation data-driven animation anatomical local model motion retargeting vibrating platform blendshapes finite element method softsmpl vertex offset

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →