← ArchivePaper2023
Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction
Vanessa Sklyarova, Jenya Chelishev, Andreea Dogaru, Igor Medvedev, Victor Lempitsky, Egor Zakharov
Two-stage strand-accurate reconstruction from monocular video or multi-view images: coarse volumetric orientation then strand optimization with learned prior.
Abstract
Generating realistic human 3D reconstructions using image or video data is essential for various communication and entertainment applications. While existing methods achieved impressive results for body and facial regions, realistic hair modeling still remains challenging due to its high mechanical complexity. This work proposes an approach capable of accurate hair geometry reconstruction at a strand level from a monocular video or multi-view images captured in uncontrolled lighting conditions. Our method has two stages, with the first stage performing joint reconstruction of coarse hair and bust shapes and hair orientation using implicit volumetric representations. The second stage then estimates a strand-level hair reconstruction by reconciling in a single optimization process the coarse volumetric constraints with hair strand and hairstyle priors learned from the synthetic data. To further increase the reconstruction fidelity, we incorporate image-based losses into the fitting process using a new differentiable renderer. The combined system, named Neural Haircut, achieves high realism and personalization of the reconstructed hairstyles. For video results, please refer to our project page †.
How to read this
- Category
- Method: strand-based hair reconstruction from images/video
- Contributions
- Strand-level hair geometry reconstruction from monocular video or multi-view images captured in uncontrolled lighting
- A two-stage pipeline: coarse joint hair/bust shape and orientation via implicit volumes, then strand optimization reconciled with priors learned from synthetic data
- A differentiable renderer with image-based losses to raise reconstruction fidelity
- Context
- Builds on learned strand-based hair modeling such as Neural Strands (Rosu 2022), extending toward in-the-wild monocular capture.Builds on: Neural Strands: Learning Hair Geometry and Appearance from Multi-View Images
- Correctness
- Demonstrated on monocular and multi-view captures; strand fidelity relies on hairstyle priors learned from synthetic data, so out-of-distribution or heavily occluded styles may be a limitation.
- Clarity
- Moderately technical; a first pass conveys the coarse-then-strand structure, a second pass is needed for the optimization and prior formulation.
- How to read it
- First pass for the two-stage strategy and where the synthetic prior enters; second pass on the optimization and differentiable rendering losses if reconstructing hair yourself.
Related work
- HAAR: Text-Conditioned Generative Model of 3D Strand-Based Human Hairstyles 2023 / arXiv
- Neural Strands: Learning Hair Geometry and Appearance from Multi-View Images 2022 / ECCV
- 3D Hair Synthesis Using Volumetric Variational Autoencoders 2018 / SIGGRAPH Asia
- Dynamic Hair Modeling from Monocular Videos Using Deep Neural Networks 2019 / SIGGRAPH Asia
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →