Online Modeling for Realtime Facial Animation

How to read this

Category

Method: real-time facial tracking via online model fitting

Contributions

• A calibration-free, training-free real-time face tracking algorithm for commodity RGB-D sensors
• Joint optimization that simultaneously solves for a detailed 3D expression model of the user and the dynamic tracking parameters
• A subspace parameterization of the dynamic facial expression space for robust, real-time computation

Context

Continues performance-based facial animation from depth input, building on Weise et al.'s Realtime Performance-Based Facial Animation by removing the user-specific training/calibration step.Builds on: Realtime Performance-Based Facial Animation

Correctness

Relies on commodity RGB-D input and on the assumption that a per-user expression model can be recovered online jointly with tracking; the authors provide an evaluation showing accurate tracking, but reader should remember robustness depends on depth-sensor quality and the chosen subspace's expressiveness.

Clarity

Clearly motivated; a first pass conveys the online-modeling idea, a second pass is needed for the joint optimization and subspace formulation.

How to read it

Focus on what the joint solve estimates (model vs. tracking parameters) and the subspace parameterization; do a second pass on the optimization to see how calibration is avoided.