← ArchivePaper2024
Learning a Generalized Physical Face Model From Data
Lingchen Yang, Gaspard Zoss, Prashanth Chandran, Markus Gross, Barbara Solenthaler, Eftychios Sifakis, Derek Bradley
Data-driven generalized physical face model that learns anatomy-consistent tissue mechanics from captured data for novel character synthesis.
Abstract
Physically-based simulation is a powerful approach for 3D facial animation as the resulting deformations are governed by physical constraints, allowing to easily resolve self-collisions, respond to external forces and perform realistic anatomy edits. Today's methods are data-driven, where the actuations for finite elements are inferred from captured skin geometry. Unfortunately, these approaches have not been widely adopted due to the complexity of initializing the material space and learning the deformation model for each character separately, which often requires a skilled artist followed by lengthy network training. In this work, we aim to make physics-based facial animation more accessible by proposing a generalized physical face model that we learn from a large 3D face dataset. Once trained, our model can be quickly fit to any unseen identity and produce a ready-to-animate physical face model automatically. Fitting is as easy as providing a single 3D face scan, or even a single face image. After fitting, we offer intuitive animation controls, as well as the ability to retarget animations across characters. All the while, the resulting animations allow for physical effects like collision avoidance, gravity, paralysis, bone reshaping and more.
How to read this
- Category
- Method: data-driven generalized physical face model
- Contributions
- A generalized physical face model learned from a large 3D face dataset that, once trained, fits quickly to any unseen identity and produces a ready-to-animate physical face model automatically.
- Fitting from minimal input (a single 3D face scan, or even a single image), avoiding per-character material-space initialization and lengthy separate training.
- Intuitive animation controls plus cross-character retargeting, while retaining physics effects like self-collision resolution and response to external forces.
- Context
- Generalizes the authors' implicit physical face model (Yang et al., 'An Implicit Physical Face Model Driven by Expression and Style') from per-character setup toward a single model learned across many identities.Builds on: An Implicit Physical Face Model Driven by Expression and Style
- Correctness
- Targets the adoption barrier of physics-based faces (per-character material setup and training); fitting from a single scan or image is a strong convenience claim, so a reader should keep in mind dependence on the training dataset's coverage and the fidelity tradeoff of generalized versus per-character material spaces.
- Clarity
- Accessible motivation; a first pass conveys the generalize-then-fit idea, a second pass for the learned material space and physical simulation formulation.
- How to read it
- First pass for why per-character setup is the bottleneck and how fitting works; second pass on the learned physical/material model if evaluating it for a production face pipeline.
Built upon by
Nothing yet.
Related work
- An Implicit Physical Face Model Driven by Expression and Style 2023 / SIGGRAPH Asia
- Phace: Physics-based Face Modeling and Animation 2017 / SIGGRAPH
- BlendForces: A Dynamic Framework for Facial Animation 2016 / CGF
- Shape Targeting: A Versatile Active Elasticity Constitutive Model 2020 / SIGGRAPH
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →