Democratizing the Creation of Animatable Facial Avatars

How to read this

Category

Preprint on building a person-specific facial animation rig without a light stage, from Fedkiw's group at Stanford.

Contributions

• A pipeline that obtains geometry, texture and enough expression information to build a person-specific animation rig with no light stage, no other high-end hardware, and no manual cleanup.
• The key trick: warp real-world photographs to align with a template avatar's geometry, then project the warped image into that template's texture, which bridges the domain gap by carrying real lighting and texture information along.
• A de-lit texture and a neutral expression geometry as by-products of that same warping and projection step.
• A repair use for the same method, improving avatars after an animation system has imported them, since those imports tend to be lossy and to hallucinate features.
• A Simon Says capture routine that records an individual performing expressions, so the default template expressions are replaced by ones that move the way that person does.

Context

No single predecessor, because the paper argues against a class of pipeline rather than against a specific method. What it is displacing is the light stage, whose lineage in this archive starts at debevec-reflectance-field-2000. The instinct it shares, that ordinary footage plus a good template can stand in for a capture stage, is the one ichim-dynamic-avatar-2015 had already pursued with hand-held video. Read it between those two.Builds on: Acquiring the Reflectance Field of a Human Face · Dynamic 3D Avatar Creation from Hand-Held Video Input

Correctness

A preprint, unreviewed, and its comparison class is pipelines whose output is the industry benchmark, so the honest question is where the quality goes rather than whether it goes. Two places to press. The warping step assumes a template close enough to the subject for the alignment to mean anything, and the paper is thin on how far that stretches. The Simon Says capture depends on the subject performing the requested expressions well, which moves a burden off the hardware and onto the person in front of the camera. Fine wrinkle and pore level detail, the actual reason light stages exist, is the thing to look for in the results and the thing least likely to be there.

Clarity

Readable and argument-forward, with the motivation stated more crisply than the method. The pipeline is described in order and is easy to follow once you get past the parenthetical asides.

How to read it

First pass: abstract and results, then decide whether the quality bar clears your use, because that decides whether the rest is worth your time. Second pass: the warping and projection step, which is the actual contribution, and the Simon Says section, which is where the rig becomes person-specific. Third pass: check whether a reviewed version has appeared since and prefer it, and if you are comparing against a scanned pipeline, put the results next to debevec-reflectance-field-2000 rather than next to a modern neural avatar.