← ArchivePaper2023
PointAvatar: Deformable Point-Based Head Avatars from Videos
Yufeng Zheng, Wang Yifan, Gordon Wetzstein, Michael J. Black, Otmar Hilliges
Deformable point cloud head avatar disentangling intrinsic albedo from normal-dependent shading, enabling FLAME-driven animation with topological flexibility.
Abstract
The ability to create realistic animatable and relightable head avatars from casual video sequences would open up wide ranging applications in communication and entertainment. Current methods either build on explicit 3D morphable meshes (3DMM) or exploit neural implicit representations. The former are limited by fixed topology, while the latter are non-trivial to deform and inefficient to render. Furthermore, existing approaches entangle lighting and albedo, limiting the ability to re-render the avatar in new environments. In contrast, we propose PointAvatar, a deformable point-based representation that disentangles the source color into intrinsic albedo and normal-dependent shading. We demonstrate that PointAvatar bridges the gap between existing mesh- and implicit representations, combining high-quality geometry and appearance with topological flexibility, ease of deformation and rendering efficiency. We show that our method is able to generate animatable 3D avatars using monocular videos from multiple sources including hand-held smartphones, laptop webcams and internet videos, achieving state-of-the-art quality in challenging cases where previous methods fail, e.g., thin hair strands, while being significantly more efficient in training than competing methods.
How to read this
- Category
- Method: deformable point-based head avatars from video
- Contributions
- PointAvatar, a deformable point-based head representation that bridges mesh and implicit approaches with topological flexibility and efficient rendering
- Disentanglement of source color into intrinsic albedo and normal-dependent shading, enabling relighting in new environments
- Animatable avatars from monocular video across sources (smartphone, webcam, internet clips), driven via FLAME
- Context
- Follows monocular neural head avatar work such as Neural Head Avatars (Grassal 2022), substituting a deformable point cloud for fixed-topology mesh or implicit volumes.Builds on: Neural Head Avatars from Monocular RGB Videos
- Correctness
- Demonstrated on monocular videos including challenging cases like thin hair; the albedo/shading split is a simplified disentanglement, so relighting and very complex appearance remain limitations to weigh.
- Clarity
- Accessible; a first pass conveys the point-based representation and its trade-offs, a second pass covers the deformation and shading model.
- How to read it
- First pass for why a point representation sits between mesh and implicit and what relighting it enables; second pass on the deformation and albedo/shading formulation if adopting it.
Builds on
Built upon by
Nothing yet.
Related work
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →