← ArchivePaper2024

Implementing a Machine Learning Deformer for CG Crowds: Our Journey

Bastien Arcelin, Sebastien Maraux, Nicolas Chaverou

DigiProIndustrial0 citesML Deformation Skinning

Production experience implementing a neural network deformer for crowd characters, covering training, integration, and quality tradeoffs.

Open the paper ↗Trace in the lineage graph

Abstract

CG crowds have become increasingly popular this last decade in the VFX and animation industry: formerly reserved to only a few high end studios and blockbusters, they are now widely used in TV shows or commercials. Yet, there is still one major limitation: in order to be ingested properly in crowd software, studio rigs have to comply with specific prerequisites, especially in terms of deformations. Usually only skinning, blend shapes and geometry caches are supported preventing close-up shots with facial performances on crowd characters. We envisioned two approaches to tackle this: either reverse engineer the hundreds of deformer nodes available in the major DCCs/plugins and incorporate them in our crowd package, or surf the machine learning wave to compress the deformations of a rig using a neural network architecture. Considering we could not commit 5+ man/years of development into this problem, and that we were excited to dip our toes in the machine learning pool, we went for the latter. From our first tests to a minimum viable product, we went through hopes and disappointments: we hit multiple pitfalls, took false shortcuts and dead ends before reaching our destination. With this paper, we hope to provide a valuable feedback by sharing the lessons we learnt from this experience.

How to read this

Category: Production talk / experience report: ML deformer for crowds
Contributions: Demonstrates a production journey implementing a neural-network deformer to compress a studio rig's deformations for CG crowd characters.
Motivates ML over reverse-engineering hundreds of DCC deformer nodes, so crowd characters can support close-up facial performance beyond skinning, blend shapes, and geometry caches.
Shares the path from first tests to a minimum viable product, including pitfalls, false shortcuts, and dead ends.
Context: Applies learned rig approximation in the spirit of Fast and Deep Deformation Approximations (Bailey et al., 'FDDA') to the specific constraints of crowd-software pipelines.Builds on: Fast and Deep Deformation Approximations
Correctness: Studio practice rather than peer-reviewed research; results are production-oriented and the value is in the reported tradeoffs and integration lessons, so treat quality and performance claims as context-specific to this studio's pipeline.
Clarity: Very accessible and narrative; a single first pass conveys the decisions and pitfalls, no formal formulation to decode.
How to read it: Read once for the practical decision rationale (ML vs node reimplementation) and the integration/quality tradeoffs; useful as a checklist of pitfalls rather than a method to reimplement.

Builds on

Fast and Deep Deformation Approximations 2018

Built upon by

Nothing yet.

Related work

Accurate Face Rig Approximation with Deep Differential Subspace Reconstruction 2020 / SIGGRAPH
Multi-Resolution Real-Time Deep Pose-Space Deformation 2024 / SIGGRAPH Asia
Dyna: A Model of Dynamic Human Shape in Motion 2015 / SIGGRAPH
Subspace Neural Physics: Fast Data-Driven Interactive Simulation 2019 / SCA

Keywords

machine learning deformer neural networks cg crowds rig approximation nonlinear deformation linear blend skinning blend shapes facial deformation differential coordinates laplacian operator principal component analysis 6d rotation representation dense neural networks convolutional neural networks graph neural network epistemic uncertainty golaem rig evaluation

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →