← ArchivePaper2015
Dynamic 3D Avatar Creation from Hand-Held Video Input
System for creating personalized animatable 3D face avatars from casual handheld video using blendshape fitting and rig transfer.
Abstract
We present a complete pipeline for creating fully rigged, personalized 3D facial avatars from hand-held video. Our system faithfully recovers facial expression dynamics of the user by adapting a blendshape template to an image sequence of recorded expressions using an optimization that integrates feature tracking, optical flow, and shape from shading. Fine-scale details such as wrinkles are captured separately in normal maps and ambient occlusion maps. From this user- and expression-specific data, we learn a regressor for on-the-fly detail synthesis during animation to enhance the perceptual realism of the avatars. Our system demonstrates that the use of appropriate reconstruction priors yields compelling face rigs even with a minimalistic acquisition system and limited user assistance. This facilitates a range of new applications in computer animation and consumer-level online communication based on personalized avatars. We present realtime application demos to validate our method.
How to read this
- Category
- Method / system: personalized facial avatar creation from video
- Contributions
- A complete pipeline producing fully rigged, personalized 3D facial avatars from hand-held video
- Blendshape-template adaptation via an optimization integrating feature tracking, optical flow, and shape from shading, with wrinkle-scale detail captured in normal and ambient-occlusion maps
- A learned regressor for on-the-fly detail synthesis during animation, demonstrated in real-time application demos
- Context
- Sits in the consumer-level facial-capture and blendshape-rig lineage, emphasizing reconstruction priors that let a minimalist acquisition setup still yield compelling rigs.
- Correctness
- Quality hinges on appropriate reconstruction priors and on the recorded expression range; with a minimalistic rig and limited user assistance, results depend on capture coverage and may degrade for expressions or details not observed in the input.
- Clarity
- Readable end-to-end system description; one pass conveys the pipeline, a second covers the optimization and detail-regressor stages.
- How to read it
- Read for the staged optimization (tracking plus flow plus shape-from-shading) and the offline-detail vs runtime-synthesis split; second pass if you need to reproduce the regressor.
Builds on
Nothing in the archive, this is a starting point.
Related work
- Example-Based Facial Rigging 2010 / SIGGRAPH
- Sketch-Based Controllers for Blendshape Facial Animation 2015 / Eurographics
- Reusable Facial Rigging and Animation: Create Once, Use Many 2007 / PhD Thesis
- Animatomy: An Animator-Centric, Anatomically Inspired System for 3D Facial Modeling, Animation and Transfer 2022 / SIGGRAPH Asia
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →