← ArchivePaper2022
MoRF: Morphable Radiance Fields for Multiview Neural Head Modeling
Daoye Wang, Prashanth Chandran, Gaspard Zoss, Derek Bradley, Paulo Gotardo
Extends NeRF into a generative morphable model producing multiview-consistent photorealistic head images with controllable identity parameters.
Abstract
Recent research work has developed powerful generative models (e.g., StyleGAN2) that can synthesize complete human head images with impressive photorealism, enabling applications such as photorealistically editing real photographs. While these models can be trained on large collections of unposed images, their lack of explicit 3D knowledge makes it difficult to achieve even basic control over 3D viewpoint without unintentionally altering identity. On the other hand, recent Neural Radiance Field (NeRF) methods have already achieved multiview-consistent, photorealistic renderings but they are so far limited to a single facial identity. In this paper, we propose a new Morphable Radiance Field (MoRF) method that extends a NeRF into a generative neural model that can realistically synthesize multiview-consistent images of complete human heads, with variable and controllable identity. MoRF allows for morphing between particular identities, synthesizing arbitrary new identities, or quickly generating a NeRF from few images of a new subject, all while providing realistic and consistent rendering under novel viewpoints. We train MoRF in a supervised fashion by leveraging a high-quality database of multiview portrait images of several people, captured in studio with polarization-based separation of diffuse and specular reflection.
How to read this
- Category
- Method: a generative morphable neural radiance field for heads
- Contributions
- MoRF, extending a NeRF into a generative neural model synthesizing multiview-consistent photorealistic complete-head images
- Variable, controllable identity: morphing between identities, synthesizing new ones, or quickly fitting a NeRF from few images
- Supervised training leveraging a high-quality dataset
- Context
- Positioned between StyleGAN2-style head generators (lacking explicit 3D control) and single-identity NeRFs, drawing on animatable face modeling (related to Feng et al.'s DECA, 2021).Builds on: Learning an Animatable Detailed 3D Face Model from In-The-Wild Images
- Correctness
- Addresses the known failure where 2D generators alter identity when changing viewpoint; the key dependency is the high-quality supervised dataset, so readers should keep in mind that identity control and novel-view consistency are tied to that training data and to the morphable parameterization.
- Clarity
- Moderately dense (NeRF plus generative morphable model); a first pass conveys the goal and capability, a second pass for the conditioning and training setup.
- How to read it
- First pass on what MoRF enables (3D-consistent identity control) versus prior GAN/NeRF tradeoffs; second pass on the morphable parameterization and supervised training if you build head avatars.
Built upon by
Nothing yet.
Related work
- NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads 2023 / SIGGRAPH
- Neural Volumes: Learning Dynamic Renderable Volumes from Images 2019 / SIGGRAPH
- Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies 2021 / CVPR
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting 2024 / CVPR
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →