← ArchivePaper2024
GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians
Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, Matthias Niessner
Rigs 3D Gaussian splats to a parametric FLAME morphable model with explicit displacement offsets, enabling fully controllable photorealistic head avatar synthesis.
Abstract
We introduce GaussianAvatars11Project page: https://shenhanqian.github.io/gaussian-avatars, a new method to create photorealistic head avatars that are fully controllable in terms of expression, pose, and viewpoint. The core idea is a dynamic 3D representation based on 3D Gaussian splats that are rigged to a parametric morphable face model. This combination facilitates photorealistic rendering while allowing for precise animation control via the underlying parametric model, e.g., through expression transfer from a driving sequence or by manually changing the morphable model parameters. We parameterize each splat by a local coordinate frame of a triangle and optimize for explicit dis-placement offset to obtain a more accurate geometric representation. During avatar reconstruction, we jointly optimize for the morphable model parameters and Gaussian splat parameters in an end-to-end fashion. We demonstrate the animation capabilities of our photorealistic avatar in several challenging scenarios. For instance, we show reen-actments from a driving video, where our method outperforms existing works by a significant margin.
How to read this
- Category
- Method: controllable photorealistic head avatars
- Contributions
- A dynamic head-avatar representation that rigs 3D Gaussian splats to a parametric morphable face model for photorealistic, fully controllable (expression, pose, viewpoint) synthesis.
- Parameterizing each splat in the local coordinate frame of a triangle and optimizing explicit displacement offsets for more accurate geometry.
- End-to-end joint optimization of morphable-model and Gaussian-splat parameters during avatar reconstruction, supporting reenactment and manual parameter control.
- Context
- Combines 3D Gaussian splatting with the FLAME parametric face model (Li et al., 'Learning a Model of Facial Shape and Expression from 4D Scans') to get rendering quality plus parametric animation control.Builds on: Learning a Model of Facial Shape and Expression from 4D Scans
- Correctness
- Demonstrated on challenging animation scenarios including reenactment from a driving video, where it reports outperforming prior work; a reader should note quality depends on the morphable-model fit and per-subject reconstruction, and the abstract's comparison claims are author-reported.
- Clarity
- Accessible if you already know Gaussian splatting and morphable models; a first pass conveys the rigging idea, a second pass for the local-frame parameterization and joint optimization.
- How to read it
- First pass for how splats are bound to a parametric face and what becomes controllable; second pass on the triangle-local frame and displacement offsets if reconstructing or driving avatars yourself.
Built upon by
Related work
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →