← ArchivePaper2020

Learning to Dress 3D People in Generative Clothing

Qianli Ma, Jinlong Yang, Anurag Ranjan, Sergi Pujades, Gerard Pons-Moll, Siyu Tang, Michael J. Black

CVPRAcademic13 cites1 descendantCFX ML Deformation

CAPE trains a conditional Mesh-VAE-GAN to generate pose-dependent clothing deformations as an additive term on SMPL, introducing the first large-scale dynamic clothed human mesh dataset.

Open the paper ↗Trace in the lineage graph

Abstract

Three-dimensional human body models are widely used in the analysis of human pose and motion. Existing models, however, are learned from minimally-clothed humans and thus do not capture the complexity of dressed humans in common images and videos. To address this, we learn a generative 3D mesh model of clothing from 3D scans of people with varying pose. Going beyond previous work, our generative model is conditioned on different clothing types, giving the ability to dress different body shapes in a variety of clothing. To do so, we train a conditional Mesh-VAE-GAN on clothing displacements from a 3D SMPL body model. This generative clothing model enables us to sample various types of clothing, in novel poses, on top of SMPL. With a focus on clothing geometry, the model captures both global shape and local structure, effectively extending the SMPL model to add clothing. To our knowledge, this is the first conditional VAE-GAN that works on 3D meshes. For clothing specifically, it is the first such model that directly dresses 3D human body meshes and generalizes to different poses.

How to read this

Category: Method plus dataset: a generative model of pose-dependent clothing on SMPL
Contributions: CAPE, a conditional Mesh-VAE-GAN that generates clothing displacements as an additive term on the SMPL body model.
Conditioning on clothing type and body shape, so different bodies can be dressed in varied clothing and posed in novel poses.
A large-scale dynamic clothed-human mesh dataset, and what the authors describe as the first conditional VAE-GAN operating directly on 3D meshes.
Context: Extends the SMPL body model (Loper et al., 'SMPL: A Skinned Multi-Person Linear Model') by adding learned clothing geometry on top of the minimally-clothed base.Builds on: SMPL: A Skinned Multi-Person Linear Model
Correctness: Clothing is modeled as displacements on SMPL, so it captures global shape and local structure but inherits SMPL's topology and may not represent loose or highly dynamic garments that depart strongly from the body; quality is bounded by the captured dataset.
Clarity: Moderately technical; a first pass conveys the additive-displacement idea, a second pass is needed for the VAE-GAN architecture and losses.
How to read it: Focus first on how clothing is posed as a residual on SMPL and what the conditioning controls; a second pass on the Mesh-VAE-GAN and the dataset pays off if you want to train or sample from it.

Builds on

SMPL: A Skinned Multi-Person Linear Model 2015

Built upon by

SMPLicit: Topology-aware Generative Model for Clothed People 2021

Related work

SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks 2021 / CVPR
3D Hair Synthesis Using Volumetric Variational Autoencoders 2018 / SIGGRAPH Asia
Learning-Based Animation of Clothing for Virtual Try-On 2019 / CGF
GarMatNet: A Learning-Based Method for Predicting 3D Garment Mesh with Parameterized Materials 2021 / MIG

Keywords

cape clothed human model smpl clothing deformation graph convolutional network mesh vae-gan patchwise discriminator pose-dependent clothing displacement layer generative model 3d body model blend skinning clothing wrinkles 4d scans latent space sampling smplify mesh autoencoder image fitting

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →