← ArchivePaper2023
Learning Neural Parametric Head Models
Simon Giebenhain, Tobias Kirschstein, Markos Georgopoulos, Martin Runz, Lourdes Agapito, Matthias Niessner
Neural parametric head model using an ensemble of local MLPs in canonical SDF space with forward deformation field, trained on 5,200+ scans from 255 subjects.
Abstract
We propose a novel 3D morphable model for complete human heads based on hybrid neural fields. At the core of our model lies a neural parametric representation that disentangles identity and expressions in disjoint latent spaces. To this end, we capture a person's identity in a canonical space as a signed distance field (SDF), and model facial expressions with a neural deformation field. In addition, our representation achieves high-fidelity local detail by introducing an ensemble of local fields centered around facial anchor points. To facilitate generalization, we train our model on a newly-captured dataset of over 3700 head scans from 203 different identities using a custom high-end 3D scanning setup. Our dataset significantly exceeds comparable existing datasets, both with respect to quality and completeness of geometry, averaging around 3.5M mesh faces per scan11We will publicly release our dataset along with a public benchmark for both neural head avatar construction as well as an evaluation on a hidden test-set for inference-time fitting.. Finally, we demonstrate that our approach outperforms state-of-the-art methods in terms of fitting error and reconstruction quality.
How to read this
- Category
- Method plus dataset: a neural parametric head model
- Contributions
- A neural 3D morphable head model disentangling identity (a canonical SDF) and expression (a neural deformation field) in separate latent spaces
- An ensemble of local fields anchored at facial points to capture high-fidelity local detail
- A newly captured high-resolution head-scan dataset and benchmark, with reported state-of-the-art fitting error and reconstruction quality
- Context
- Extends implicit 3D morphable head modeling (i3DMM) by moving to hybrid neural fields with local anchor-based detail rather than a single global field.Builds on: i3DMM: Deep Implicit 3D Morphable Model of Human Heads
- Correctness
- Trained on a large custom multi-subject scan set captured with a high-end rig; the note and abstract give slightly different scan/subject counts, so verify exact dataset numbers in the paper, and expect fitting quality to reflect the capture rig's distribution.
- Clarity
- Clearly structured; a first pass conveys the identity/expression disentanglement and local-field idea, a second pass is needed for the SDF and deformation-field formulation.
- How to read it
- First pass for the representation and the dataset/benchmark contribution; second pass on the local-field ensemble and canonical-space training if fitting your own heads.
Builds on
Built upon by
Nothing yet.
Related work
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →