← ArchivePaper2021
SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks
Converts raw 3D scans of clothed humans into animatable avatars using locally pose-aware implicit functions for pose-dependent correctives without mesh registration.
Abstract
We present SCANimate, an end-to-end trainable framework that takes raw 3D scans of a clothed human and turns them into an animatable avatar. These avatars are driven by pose parameters and have realistic clothing that moves and deforms naturally. SCANimate does not rely on a customized mesh template or surface mesh registration. We observe that fitting a parametric 3D body model, like SMPL, to a clothed human scan is tractable while surface registration of the body topology to the scan is often not, because clothing can deviate significantly from the body shape. We also observe that articulated transformations are invertible, resulting in geometric cycle-consistency in the posed and unposed shapes. These observations lead us to a weakly supervised learning method that aligns scans into a canonical pose by disentangling articulated deformations without template-based surface registration. Furthermore, to complete missing regions in the aligned scans while modeling pose-dependent deformations, we introduce a locally pose-aware implicit function that learns to complete and model geometry with learned pose correctives. In contrast to commonly used global pose embeddings, our local pose conditioning significantly reduces long-range spurious correlations and improves generalization to unseen poses, especially when training data is limited.
How to read this
- Category
- Method: weakly supervised animatable clothed avatars from scans
- Contributions
- End-to-end framework turning raw 3D scans of clothed humans into pose-driven animatable avatars without a custom mesh template or surface registration
- Uses geometric cycle-consistency of invertible articulated transformations to canonicalize scans in a weakly supervised way
- Introduces a locally pose-aware implicit function that completes missing regions and models pose-dependent deformations via learned local pose correctives
- Context
- Builds on fitting the SMPL parametric body (Loper et al.) to scans, then departs from template surface registration by learning local pose-aware implicit correctives instead of a global pose embedding.Builds on: SMPL: A Skinned Multi-Person Linear Model
- Correctness
- Relies on the observations that SMPL fitting to clothed scans is tractable and articulated transforms are invertible (cycle-consistent), trained weakly supervised; clothing that deviates far from the body or scan completion in unseen regions are where the learned correctives are most stretched, so a reader should keep generalization limits in mind.
- Clarity
- Conceptually rich; a first pass conveys the canonicalization-via-cycle-consistency and local-implicit-corrective ideas, a second pass is needed for the disentanglement and implicit-function formulation.
- How to read it
- First pass for the registration-free canonicalization and local-pose-corrective insight; do a second pass on the cycle-consistency loss and local implicit function if you build avatars from scans.
Builds on
Built upon by
Nothing yet.
Related work
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →