← ArchivePaper2022
FaceVerse: A Fine-Grained and Detail-Controllable 3D Face Morphable Model from a Hybrid Dataset
Lizhen Wang, Zhiyuan Chen, Tao Yu, Chenguang Ma, Liang Li, Yebin Liu
Coarse-to-fine 3DMM built from 60K RGB-D images and 2K high-fidelity scans, with controllable StyleGAN-based detail generation for both base and fine modules.
Abstract
We present FaceVerse, a fine-grained 3D Neural Face Model, which is built from hybrid East Asian face datasets containing 60K fused RGB-D images and 2K high-fidelity 3D head scan models. A novel coarse-to-fine structure is proposed to take better advantage of our hybrid dataset. In the coarse module, we generate a base parametric model from large-scale RGB-D images, which is able to predict accurate rough 3D face models in different genders, ages, etc. Then in the fine module, a conditional StyleGAN architecture trained with high-fidelity scan models is introduced to enrich elaborate facial geometric and texture details. Note that different from previous methods, our base and detailed modules are both changeable, which enables an innovative application of adjusting both the basic attributes and the facial details of 3D face models. Furthermore, we propose a single-image fitting framework based on differentiable rendering. Rich experiments show that our method outperforms the state-of-the-art methods.
How to read this
- Category
- Method / model: a detail-controllable 3D face morphable model
- Contributions
- A coarse-to-fine 3D neural face model built from a hybrid dataset of 60K fused RGB-D images and 2K high-fidelity 3D head scans
- A coarse base parametric model plus a fine module using a conditional StyleGAN to enrich geometric and texture detail, with both modules independently adjustable
- A single-image fitting framework based on differentiable rendering
- Context
- A 3D morphable model in the data-driven, riggable-face tradition of Yang et al.'s FaceScape, here combining RGB-D capture with high-fidelity scans and a generative detail module.Builds on: FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction
- Correctness
- The hybrid dataset is built from East Asian faces, so demographic coverage is a stated scope to keep in mind; reported gains over prior methods come from experiments and the differentiable-rendering fit, and the conditional-StyleGAN detail is generative (plausible) rather than measured ground truth.
- Clarity
- Accessible; a first pass conveys the coarse-to-fine design and the changeable-modules idea, a second pass for the fitting and StyleGAN conditioning.
- How to read it
- First pass for the two-module architecture and what the hybrid dataset buys you; second pass if you need the differentiable single-image fitting or want to reuse the model, noting the dataset demographics.
Builds on
Built upon by
Nothing yet.
Related work
- EMOCA: Emotion Driven Monocular Face Capture and Animation 2022 / CVPR
- Neural Volumes: Learning Dynamic Renderable Volumes from Images 2019 / SIGGRAPH
- Deep Appearance Models for Face Rendering 2018 / SIGGRAPH
- FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction 2020 / CVPR
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →