← ArchivePaper2021

i3DMM: Deep Implicit 3D Morphable Model of Human Heads

Tarun Yenamandra, Ayush Tewari, Florian Bernard, Hans-Peter Seidel, Mohamed Elgharib, Daniel Cremers, Christian Theobalt

CVPRAcademic157 cites1 descendantFacial

First deep implicit 3DMM of full heads including hair, using signed distance functions and disentangled geometry and color latent spaces.

Open the paper ↗Trace in the lineage graph

Abstract

We present the first deep implicit 3D morphable model (i3DMM) of full heads. Unlike earlier morphable face models it not only captures identity-specific geometry, texture, and expressions of the frontal face, but also models the entire head, including hair. We collect a new dataset consisting of 64 people with different expressions and hairstyles to train i3DMM. Our approach has the following favorable properties: (i) It is the first full head morphable model that includes hair. (ii) In contrast to mesh-based models it can be trained on merely rigidly aligned scans, without requiring difficult non-rigid registration. (iii) We design a novel architecture to decouple the shape model into an implicit reference shape and a deformation of this reference shape. With that, dense correspondences between shapes can be learned implicitly. (iv) This architecture allows us to semantically disentangle the geometry and color components, as color is learned in the reference space. Geometry is further disentangled as identity, expressions, and hairstyle, while color is disentangled as identity and hairstyle components. We show the merits of i3DMM using ablation studies, comparisons to state-of-the-art models, and applications such as semantic head editing and texture transfer. We will make our model publicly available1.

How to read this

Category: Method / model: deep implicit 3D morphable model of full heads
Contributions: The first deep implicit 3D morphable model of full heads, including hair, captured with signed distance functions
An architecture that decouples shape into an implicit reference shape plus a deformation, learning dense correspondences implicitly and training on merely rigidly aligned scans
Semantic disentanglement of geometry (identity, expression, hairstyle) and color (identity, hairstyle), supported by a new 64-person dataset
Context: Extends the morphable-model line to implicit representations, building on the classic mesh-based morphable face model of Blanz and Vetter.Builds on: A Morphable Model for the Synthesis of 3D Faces
Correctness: Trained on a new dataset of 64 people with varied expressions and hairstyles and supported by ablations and comparisons; the modest dataset size and the implicit/SDF representation are the scope limits a reader should keep in mind.
Clarity: Technical (CVPR) but well-motivated; a first pass conveys the model and disentanglement, a second pass is needed for the architecture.
How to read it: Read for the reference-shape-plus-deformation design and the geometry/color disentanglement; second pass on the architecture and dataset if you build face/head models.

Builds on

A Morphable Model for the Synthesis of 3D Faces 1999

Built upon by

Learning Neural Parametric Head Models 2023

Related work

Learning Neural Parametric Head Models 2023 / CVPR
Learning a Model of Facial Shape and Expression from 4D Scans 2017 / SIGGRAPH Asia
3D Morphable Face Models: Past, Present and Future 2021 / SIGGRAPH
EMOCA: Emotion Driven Monocular Face Capture and Animation 2022 / CVPR

Keywords

i3dmm 3d morphable model deep implicit surface signed distance function human heads reference shape network shape deformation network color network autodecoder dense correspondences disentanglement hairstyle flame basel face model positional encoding texture transfer semantic head editing photogrammetric scans

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →