← ArchivePaper2024
Compressed Skinning for Facial Blendshapes
Ladislav Kavan, John Doublestein, Martin Prazak, Matthew Cioffi, Doug Roble
Proximal-algorithm method to bake large facial blendshape sets into a compact linear blend skinning representation for real-time use.
Abstract
We present a new method to bake classical facial animation blendshapes into a fast linear blend skinning representation. Previous work explored skinning decomposition methods that approximate general animated meshes using a dense set of bone transformations; these optimizers typically alternate between optimizing for the bone transformations and the skinning weights. We depart from this alternating scheme and propose a new approach based on proximal algorithms, which effectively means adding a projection step to the popular Adam optimizer. This approach is very flexible and allows us to quickly experiment with various additional constraints and/or loss functions. Specifically, we depart from the classical skinning paradigms and restrict the transformation coefficients to contain only about 90% non-zeros, while achieving similar accuracy and visual quality as the state-of-the-art. The sparse storage enables our method to deliver significant savings in terms of both memory and run-time speed. We include a compact implementation of our new skinning decomposition method in PyTorch, which is easy to experiment with and modify to related problems.
How to read this
- Category
- Method: a skinning-decomposition algorithm for compressing facial blendshapes
- Contributions
- A method to bake classical facial blendshapes into a fast linear blend skinning representation
- A proximal-algorithm approach (a projection step added to Adam) replacing the usual alternating optimization, allowing flexible constraints and losses
- Sparse transformation coefficients (about 90% non-zeros restricted out) yielding memory and run-time savings at comparable accuracy, with a compact PyTorch implementation
- Context
- Sits in the skinning-decomposition and pose-space deformation lineage (building on Lewis et al.'s Pose Space Deformation), targeting real-time facial animation.Builds on: Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation
- Correctness
- Claims similar accuracy and visual quality to state-of-the-art while enforcing sparsity; keep in mind it approximates blendshapes with LBS, so reproduction quality depends on the target rig and the sparsity level chosen, and gains are reported relative to prior decomposition methods.
- Clarity
- Accessible to readers familiar with skinning and optimization; a first pass conveys the proximal/projection idea, a second pass for the loss formulation and constraints.
- How to read it
- Focus on how the proximal projection enforces sparsity and why that helps memory and speed; a second pass plus the PyTorch code pays off if implementing or extending it.
Builds on
Built upon by
Nothing yet.
Related work
- Easy Rigging of Face by Automatic Registration and Transfer of Skinning Parameters 2010 / ICCVG
- Fast and Efficient Skinning of Animated Meshes 2010 / CGF
- Robust and Accurate Skeletal Rigging from Mesh Sequences 2014 / SIGGRAPH
- Efficient Dynamic Skinning with Low-Rank Helper Bone Controllers 2016 / SIGGRAPH
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →