← ArchivePaper2019
High-Quality Face Capture Using Anatomical Muscles
Makes a muscle-based facial system fully differentiable by coupling it with a blendshape basis, enabling both optimization and learning-based performance capture.
Abstract
Muscle-based systems have the potential to provide both anatomical accuracy and semantic interpretability as compared to blendshape models; however, a lack of expressivity and differentiability has limited their impact. Thus, we propose modifying a recently developed rather expressive muscle-based system in order to make it fully-differentiable; in fact, our proposed modifications allow this physically robust and anatomically accurate muscle model to conveniently be driven by an underlying blendshape basis. Our formulation is intuitive, natural, as well as monolithically and fully coupled such that one can differentiate the model from end to end, which makes it viable for both optimization and learning-based approaches for a variety of applications. We illustrate this with a number of examples including both shape matching of three-dimensional geometry as as well as the automatic determination of a three-dimensional facial pose from a single two-dimensional RGB image without using markers or depth information.
How to read this
- Category
- Method: a differentiable anatomical-muscle facial model for capture
- Contributions
- Modifies an expressive muscle-based facial system to be fully differentiable so it can be driven by an underlying blendshape basis
- A monolithically and fully coupled formulation that is differentiable end to end, making it usable for both optimization and learning-based approaches
- Demonstrations including 3D shape matching and automatic 3D facial pose from a single 2D RGB image without markers or depth
- Context
- Builds on anatomical face simulation (Cong et al.), combining muscle-based physical accuracy with the differentiability and control of blendshape models.Builds on: Fully Automatic Generation of Anatomical Face Simulation Models
- Correctness
- Couples a physically robust muscle model to a blendshape basis; the single-image pose result is markerless and depth-free, so reconstruction quality depends on the muscle model's expressivity and the blendshape coupling, and monocular ambiguity remains a caveat.
- Clarity
- Technical; a first pass conveys why differentiability matters, while the coupled formulation needs a careful second pass.
- How to read it
- First pass for the motivation (muscle accuracy plus blendshape control plus differentiability); second pass on the coupling and differentiation if integrating into an optimization or learning pipeline.
Built upon by
Nothing yet.
Related work
- Art-Directed Muscle Simulation for High-End Facial Animation 2016 / SCA
- Lessons from the Evolution of an Anatomical Facial Muscle Model 2017 / DigiPro
- Fully Automatic Generation of Anatomical Face Simulation Models 2015 / SCA
- Animatomy: An Animator-Centric, Anatomically Inspired System for 3D Facial Modeling, Animation and Transfer 2022 / SIGGRAPH Asia
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →