← ArchivePaper2020
Single-Shot High-Quality Facial Geometry and Skin Appearance Capture
Jeremy Riviere, Paulo Gotardo, Derek Bradley, Abhijeet Ghosh, Thabo Beeler
Single-shot facial capture combining polarized spherical gradient illumination with stereo to recover geometry and skin reflectance simultaneously.
Abstract
We propose a new light-weight face capture system capable of reconstructing both high-quality geometry and detailed appearance maps from a single exposure. Unlike currently employed appearance acquisition systems, the proposed technology does not require active illumination and hence can readily be integrated with passive photogrammetry solutions. These solutions are in widespread use for 3D scanning humans as they can be assembled from off-the-shelf hardware components, but lack the capability of estimating appearance. This paper proposes a solution to overcome this limitation, by adding appearance capture to photogrammetry systems. The only additional hardware requirement to these solutions is that a subset of the cameras are cross-polarized with respect to the illumination, and the remaining cameras are parallel-polarized. The proposed algorithm leverages the images with the two different polarization states to reconstruct the geometry and to recover appearance properties. We do so by means of an inverse rendering framework, which solves per texel diffuse albedo, specular intensity, and high-resolution normals, as well as global specular roughness considering the subsurface scattering nature of skin.
How to read this
- Category
- Method / capture system: single-shot facial geometry and appearance acquisition
- Contributions
- A light-weight face capture system that reconstructs both high-quality geometry and detailed appearance maps from a single exposure, without active illumination
- Adds appearance capture to passive photogrammetry rigs by cross-polarizing a subset of cameras and parallel-polarizing the rest
- An inverse-rendering framework solving per-texel diffuse albedo, specular intensity, high-resolution normals, and a global specular roughness
- Context
- Extends single-shot passive photogrammetry capture (in the lineage of Beeler et al.'s High-Quality Single-Shot Capture of Facial Geometry) by folding appearance estimation into the same off-the-shelf, passive rig.Builds on: High-Quality Single-Shot Capture of Facial Geometry
- Correctness
- Demonstrated as an inverse-rendering recovery that depends on the cross/parallel-polarization split and an assumed reflectance model (diffuse, specular intensity, global roughness, subsurface scattering); a reader should note that a single global roughness and the polarization assumptions bound how much spatial appearance variation can be recovered.
- Clarity
- Accessible at the system level; a first pass conveys the polarization-plus-stereo idea, a second pass is needed for the inverse-rendering formulation.
- How to read it
- First pass for the hardware setup (who is cross- vs parallel-polarized) and which maps come out; do a second pass on the inverse-rendering objective if you intend to reproduce the appearance solve.
Builds on
Built upon by
Nothing yet.
Related work
- Creating an Actor-Specific Facial Rig from Performance Capture 2016 / DigiPro
- Acquiring the Reflectance Field of a Human Face 2000 / SIGGRAPH
- FaceScape: A Large-Scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction 2020 / CVPR
- The Digital Emily Project: Achieving a Photorealistic Digital Actor 2010 / Tech Note
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →