← ArchivePaper2024
Anatomically Constrained Implicit Face Models
Implicit face model constrained by anatomical priors, improving 3D face fitting generalization across diverse identities and expressions.
Abstract
Coordinate based implicit neural representations have gained rapid popularity in recent years as they have been successfully used in image, geometry and scene modeling tasks. In this work, we present a novel use case for such implicit representations in the context of learning anatomi-cally constrained face models. Actor specific anatomically constrainedface models are the state of the art in bothfacial performance capture and performance retargeting. Despite their practical success, these anatomical models are slow to evaluate and often require extensive data capture to be built. We propose the anatomical implicit face model; an ensem-ble of implicit neural networks that jointly learn to model the facial anatomy and the skin surface with high-fidelity, and can readily be used as a drop in replacement to con-ventional blendshape models. Given an arbitrary set of skin surface meshes of an actor and only a neutral shape with estimated skull and jaw bones, our method can recover a dense anatomical substructure which constrains every point on the facial surface. We demonstrate the usefulness of our approach in several tasks ranging from shape fitting, shape editing, and performance retargeting.
How to read this
- Category
- Method: an anatomically constrained implicit neural face model
- Contributions
- An ensemble of implicit neural networks that jointly model facial anatomy and skin surface as a drop-in replacement for blendshape models
- Recovery of a dense anatomical substructure constraining every surface point from skin meshes plus a neutral shape with estimated skull and jaw
- Demonstrated use across shape fitting, shape editing and performance retargeting
- Context
- Brings coordinate-based implicit neural representations into actor-specific anatomically constrained face modeling, building on prior anatomical local deformation work such as Wu et al.'s monocular face capture model.Builds on: An Anatomically Constrained Local Deformation Model for Monocular Face Capture
- Correctness
- Validated on tasks like fitting, editing and retargeting; note it still requires a neutral shape with estimated skull and jaw bones as input, and the anatomical substructure is learned/recovered rather than measured, so reconstructed internals are an approximation.
- Clarity
- Moderately accessible if you know implicit neural fields and blendshapes; a first pass gives the motivation, a second pass for the network ensemble and constraint formulation.
- How to read it
- Focus on how the implicit model replaces classical anatomical blendshapes and what inputs it needs; a second pass pays off for the training setup and how anatomical constraints are enforced.
Built upon by
Nothing yet.
Related work
- Animatomy: An Animator-Centric, Anatomically Inspired System for 3D Facial Modeling, Animation and Transfer 2022 / SIGGRAPH Asia
- Learning an Animatable Detailed 3D Face Model from In-The-Wild Images 2021 / SIGGRAPH
- Neural Head Avatars from Monocular RGB Videos 2022 / CVPR
- PointAvatar: Deformable Point-Based Head Avatars from Videos 2023 / CVPR
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →