← ArchivePaper2025
RigAnyFace: Scaling Neural Facial Mesh Auto-Rigging with Unlabeled Data
Wenchao Ma, Dario Kneubuehler, Maurice Chu, Ian Sachs, Haomiao Jiang, Sharon X. Huang
A triangulation-agnostic surface network deforms neutral facial meshes into industry-standard FACS poses, using 2D supervision on unlabeled meshes to scale beyond scarce professional-rigged data.
Abstract
In this paper, we present RigAnyFace (RAF), a scalable neural auto-rigging framework for facial meshes of diverse topologies, including those with multiple disconnected components. RAF deforms a static neutral facial mesh into industry-standard FACS poses to form an expressive blendshape rig. Deformations are predicted by a triangulation-agnostic surface learning network augmented with our tailored architecture design to condition on FACS parameters and efficiently process disconnected components. For training, we curated a dataset of facial meshes, with a subset meticulously rigged by professional artists to serve as accurate 3D ground truth for deformation supervision. Due to the high cost of manual rigging, this subset is limited in size, constraining the generalization ability of models trained exclusively on it. To address this, we design a 2D supervision strategy for unlabeled neutral meshes without rigs. This strategy increases data diversity and allows for scaled training, thereby enhancing the generalization ability of models trained on this augmented data. Extensive experiments demonstrate that RAF is able to rig meshes of diverse topologies on not only our artist-crafted assets but also in-the-wild samples, outperforming previous works in accuracy and generalizability.
How to read this
- Category
- Method: neural facial auto-rigging scaled with unlabeled data
- Contributions
- RigAnyFace, a triangulation-agnostic surface network that deforms a neutral facial mesh into FACS blendshape poses across diverse topologies, including disconnected components
- An architecture conditioned on FACS parameters that efficiently handles meshes with multiple disconnected parts
- A 2D supervision strategy for unlabeled neutral meshes that scales training beyond a small artist-rigged ground-truth subset
- Context
- Follows automatic facial rig/asset generation work such as Dynamic Facial Asset and Rig Generation from a Single Scan (Li et al. 2020), addressing the scarcity of professionally rigged 3D faces.Builds on: Dynamic Facial Asset and Rig Generation from a Single Scan
- Correctness
- Generalization is demonstrated across topologies, but the approach leans on a limited artist-rigged set augmented by 2D supervision, so the 2D signal is the key assumption and its fidelity bounds the realism of out-of-distribution rigs.
- Clarity
- Method-paper density; a first pass conveys the data-scaling motivation, a second pass needed for the network and the 2D supervision losses.
- How to read it
- Read first for why 2D supervision is used and how disconnected components are handled; second pass on the surface-learning architecture and supervision losses if you weigh it against artist-built rigs.
Built upon by
Nothing yet.
Related work
- Creating an Actor-Specific Facial Rig from Performance Capture 2016 / DigiPro
- Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild 2023 / SIGGRAPH
- Dynamic 3D Avatar Creation from Hand-Held Video Input 2015 / SIGGRAPH
- Dynamic Facial Asset and Rig Generation from a Single Scan 2020 / SIGGRAPH Asia
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →