← ArchivePaper2020

Fast and Deep Facial Deformations

Stephen W. Bailey, Dalton Omens, Paul DiLorenzo, James F. O'Brien

SIGGRAPHIndustrial52 citesFacial ML Deformation

Fast neural network deformer for production facial rigs approximating complex corrective shapes with real-time performance.

Open the paper ↗Trace in the lineage graph

Abstract

Film-quality characters typically display highly complex and expressive facial deformation. The underlying rigs used to animate the deformations of a character's face are often computationally expensive, requiring high-end hardware to deform the mesh at interactive rates. In this paper, we present a method using convolutional neural networks for approximating the mesh deformations of characters' faces. For the models we tested, our approximation runs up to 17 times faster than the original facial rig while still maintaining a high level of fidelity to the original rig. We also propose an extension to the approximation for handling high-frequency deformations such as fine skin wrinkles. While the implementation of the original animation rig depends on an extensive set of proprietary libraries making it difficult to install outside of an in-house development environment, our fast approximation relies on the widely available and easily deployed TensorFlow libraries. In addition to allowing high frame rate evaluation on modest hardware and in a wide range of computing environments, the large speed increase also enables interactive inverse kinematics on the animation rig. We demonstrate our approach and its applicability through interactive character posing and real-time facial performance capture.

How to read this

Category: Method: neural deformer for production facial rigs
Contributions: A convolutional-network approximation of facial mesh deformations running far faster than the original rig at high fidelity
An extension that recovers high-frequency deformations such as fine skin wrinkles
A TensorFlow-based, easily deployed approximation that enables interactive inverse kinematics on the animation rig
Context: Extends Bailey et al.'s Fast and Deep Deformation Approximations to faces and is informed by production deformation systems such as the DreamWorks facial motion and deformation work.Builds on: Fast and Deep Deformation Approximations · DreamWorks Animation Facial Motion and Deformation System
Correctness: Speedups and fidelity are demonstrated on the specific rigs tested, so the reported up-to-17x figure and quality are per-model; as with any learned approximation, behavior outside the sampled rig-control space and on unseen characters is not guaranteed.
Clarity: Accessible; a first pass conveys the speed/portability payoff, a second pass for the CNN architecture and the wrinkle extension.
How to read it: First pass for the motivation (interactive evaluation off proprietary libraries); second pass on the network, the high-frequency extension, and the IK use case if you need real-time rig evaluation.

Builds on

Built upon by

Nothing yet.

Related work

Fast and Deep Deformation Approximations 2018 / SIGGRAPH
Accurate Face Rig Approximation with Deep Differential Subspace Reconstruction 2020 / SIGGRAPH
Production-Level Facial Performance Capture Using Deep Convolutional Neural Networks 2017 / SCA
Direct Manipulation Blendshapes 2010 / TOG

Keywords

facial deformation convolutional neural network mesh deformation character rig rig approximation deformation maps texture coordinates uv map inverse kinematics feed-forward neural network blendshapes linear blend skinning pose space deformation deep learning rig function inversion facial performance capture k-means clustering tensorflow

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →