← ArchivePaper2023
An Implicit Physical Face Model Driven by Expression and Style
Lingchen Yang, Gaspard Zoss, Prashanth Chandran, Paulo Gotardo, Markus Gross, Barbara Solenthaler, Eftychios Sifakis, Derek Bradley
Implicit neural face model coupling expression controls with physics simulation, enabling expression-driven and style-varied realistic facial deformation.
Abstract
3D facial animation is often produced by manipulating facial deformation models (or rigs), that are traditionally parameterized by expression controls. A key component that is usually overlooked is expression “style", as in, how a particular expression is performed. Although it is common to define a semantic basis of expressions that characters can perform, most characters perform each expression in their own style. To date, style is usually entangled with the expression, and it is not possible to transfer the style of one character to another when considering facial animation. We present a new face model, based on a data-driven implicit neural physics model, that can be driven by both expression and style separately. At the core, we present a framework for learning implicit physics-based actuations for multiple subjects simultaneously, trained on a few arbitrary performance capture sequences from a small set of identities. Once trained, our method allows generalized physics-based facial animation for any of the trained identities, extending to unseen performances. Furthermore, it grants control over the animation style, enabling style transfer from one character to another or blending styles of different characters.
How to read this
- Category
- Method: an implicit physics-based face model
- Contributions
- A data-driven implicit neural physics face model driven separately by expression and by style
- A framework for learning implicit physics-based actuations for multiple subjects simultaneously from a few capture sequences
- Generalized physics-based facial animation across trained identities, with style transfer and blending between characters
- Context
- Builds on physically enriched facial rigs such as Kozlov's Enriching Facial Blendshape Rigs with Physical Simulation, disentangling expression from performance style within a learned implicit physics model.Builds on: Enriching Facial Blendshape Rigs with Physical Simulation
- Correctness
- Trained on a few arbitrary performance-capture sequences from a small identity set and claimed to extend to unseen performances; generalization beyond the trained identities and the limited training data is the key caveat a reader should keep in mind.
- Clarity
- Conceptually accessible on the expression-versus-style split; the implicit physics and actuation learning warrant a second pass.
- How to read it
- Focus on how expression and style are disentangled and what the implicit actuation represents; a second/third pass on the physics learning is worthwhile if you work on facial rigs or style transfer.
Built upon by
Related work
- Learning a Generalized Physical Face Model From Data 2024 / SIGGRAPH
- Phace: Physics-based Face Modeling and Animation 2017 / SIGGRAPH
- BlendForces: A Dynamic Framework for Facial Animation 2016 / CGF
- High-Quality Face Capture Using Anatomical Muscles 2019 / CVPR
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →