← ArchivePaper2014

Facial Retargeting Using Neural Networks

Timothy Costigan, Mukta Prasad, Rachel McDonnell

Learns face-motion retargeting from actor to virtual model using MLP and RBF neural networks trained on small datasets.

Open the paper ↗Trace in the lineage graph

Abstract

This paper presents a system that maps optical motion capture markers from an actor's face to blendshape weights on a virtual model through supervised learning of a small training dataset. The authors compare Radial Basis Function Networks (RBFNs), previously used for this task, against Multi-Layer Perceptron Artificial Neural Networks (ANNs), which to their knowledge had not been applied to marker-to-blendshape retargeting. The face is broken into upper face, eye, and lower face regions, with PCA reduction applied per region before learning the mapping. Their results found that both systems produced similar output, with the ANN sometimes proving more expressive but harder to train and more sensitive to preprocessing than the RBFN.

How to read this

Category: Method: marker-to-blendshape facial retargeting via neural networks
Contributions: A supervised system mapping an actor's optical mocap face markers to blendshape weights on a virtual model, trained on a small dataset
A comparison of Multi-Layer Perceptron ANNs against the previously used Radial Basis Function Networks for this marker-to-blendshape task
A regional decomposition (upper face, eye, lower face) with per-region PCA reduction before learning the mapping
Context: Builds on artist-friendly performance-driven retargeting (Seol et al.'s 'Artist Friendly Facial Animation Retargeting'), substituting learned MLP/RBF regressors for the marker-to-rig mapping.Builds on: Artist Friendly Facial Animation Retargeting
Correctness: An empirical comparison: the authors report both methods produce similar output, with the ANN sometimes more expressive but harder to train and more sensitive to preprocessing; conclusions rest on a small training set, so generalization beyond their data should not be assumed.
Clarity: Accessible and short; a single first pass conveys the setup and findings, with a second pass only if you want the regional PCA and network configuration details.
How to read it: Read mainly for the practical takeaway (ANN vs RBFN trade-offs); focus on the training-data size, regional split, and preprocessing sensitivity rather than expecting a definitive winner.

Builds on

Artist Friendly Facial Animation Retargeting 2011

Built upon by

Nothing yet.

Related work

Facial Retargeting with Automatic Range of Motion Alignment 2017 / SIGGRAPH
FDLS: A Deep Learning Approach to Production Quality, Controllable, and Retargetable Facial Performances 2022 / DigiPro
High Fidelity Facial Animation Capture and Retargeting with Contours 2013 / SCA
Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild 2023 / SIGGRAPH

Keywords

facial retargeting radial basis function network multi-layer perceptron artificial neural network blendshape weights optical motion capture markers supervised learning principal component analysis backpropagation facial action coding system multiquadric basis function sigmoid activation mean squared error iterative closest point facial regions performance-driven facial animation

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →