← ArchivePaper2020

Investigating Perceptually Based Models to Predict Importance of Facial Blendshapes

Emma Carrigan, Katja Zibrek, Rozenn Dahyot, Rachel McDonnell

MIGAcademic11 citesFacial

Perceptual study and predictive model ranking facial blendshape importance for games to enable efficient rig compression; won MIG 2020 Best Short Paper.

Open the paper ↗Trace in the lineage graph

Abstract

Blendshape facial rigs are used extensively in the industry for facial animation of virtual humans. However, storing and manipulating large numbers of facial meshes is costly in terms of memory and computation for gaming applications, yet the relative perceptual importance of blendshapes has not yet been investigated. Research in Psychology and Neuroscience has shown that our brains process faces differently than other objects, so we postulate that the perception of facial expressions will be feature-dependent rather than based purely on the amount of movement required to make the expression. In this paper, we explore the noticeability of blendshapes under different activation levels, and present new perceptually based models to predict perceptual importance of blendshapes. The models predict visibility based on commonly-used geometry and image-based metrics.

How to read this

Category: Perceptual study plus a predictive model for blendshape importance
Contributions: A perceptual study of how noticeable individual facial blendshapes are at different activation levels.
New perceptually based models that predict the importance (visibility) of blendshapes from common geometry and image-based metrics.
A basis for rig compression in games, ranking blendshapes by perceptual rather than purely geometric significance.
Context: Relates to direct-manipulation blendshape rigs (Lewis and Anjyo's 'Direct Manipulation Blendshapes') and to psychology/neuroscience findings that faces are processed differently from other objects.Builds on: Direct Manipulation Blendshapes
Correctness: Models are grounded in a human perceptual study, so conclusions are tied to the stimuli, expressions, and viewing conditions tested; predictions may not transfer to faces or rigs far from those used in the study (it is a short paper, so scope is limited).
Clarity: Accessible; a first pass conveys the perceptual premise and the use case, a second pass clarifies the metrics behind the model.
How to read it: Read for the motivation and the chosen perceptual metrics; one careful pass is usually enough, with a second pass only if you intend to apply the ranking to compress a specific rig.

Builds on

Direct Manipulation Blendshapes 2010

Built upon by

Nothing yet.

Related work

Animating Facial Expressions 1981 / SIGGRAPH
A Muscle Model for Animating Three-Dimensional Facial Expression 1987 / SIGGRAPH
FaceWarehouse: A 3D Facial Expression Database for Visual Computing 2014 / TVCG
Expression Packing: As-Few-As-Possible Training Expressions for Blendshape Transfer 2020 / CGF

Keywords

facial blendshapes action units facial action coding system perceptual importance blendshape reduction face perception generalized linear model poisson distribution akaike information criterion root-mean-square error spatio-temporal edge difference structural similarity index mean-squared-error perceptual error metric blendshape transfer level-of-detail virtual humans activation levels

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →