← ArchivePaper2022

SNUG: Self-Supervised Neural Dynamic Garments

Igor Santesteban, Miguel A. Otaduy, Dan Casas

Physics-based self-supervised loss recasts implicit integration as optimization, training garment networks without labeled data at two orders of magnitude faster than supervised methods.

Open the paper ↗Trace in the lineage graph

Abstract

We present a self-supervised method to learn dynamic 3D deformations of garments worn by parametric human bodies. State-of-the-art data-driven approaches to model 3D garment deformations are trained using supervised strategies that require large datasets, usually obtained by expensive physics-based simulation methods or professional multi-camera capture setups. In contrast, we propose a new training scheme that removes the need for ground-truth samples, enabling self-supervised training of dynamic 3D garment deformations. Our key contribution is to realize that physics-based deformation models, traditionally solved in a frame-by-frame basis by implicit integrators, can be recasted as an optimization problem. We leverage such optimization-based scheme to formulate a set of physics-based loss terms that can be used to train neural networks without precomputing ground-truth data. This allows us to learn models for interactive garments, including dynamic deformations and fine wrinkles, with a two orders of magnitude speed up in training time compared to state-of-the-art supervised methods.

How to read this

Category: Method: self-supervised neural garment deformation
Contributions: A self-supervised scheme that learns dynamic 3D garment deformation on parametric bodies without ground-truth data
Recasting implicit-integrator physics as an optimization, yielding physics-based loss terms to train the network directly
Interactive garments with dynamics and fine wrinkles, with a reported two-orders-of-magnitude training speedup over supervised methods
Context: Departs from supervised, simulation- or capture-trained garment models such as Santesteban et al. (Learning-Based Animation of Clothing for Virtual Try-On, 2019) by replacing labeled data with a physics-derived loss.Builds on: Learning-Based Animation of Clothing for Virtual Try-On
Correctness: The core assumption is that frame-by-frame implicit integration can be written as an optimization whose terms serve as a training loss; results are demonstrated on garments over parametric human bodies, so behavior outside the trained body and garment space is the main caution.
Clarity: Moderately dense; the idea is graspable on a first pass, but the loss derivation from the implicit integrator rewards a second pass.
How to read it: Read once for the self-supervised insight (physics as loss); do a second pass on the energy terms and how implicit integration becomes the optimization objective if you want to apply it to other deformables.

Builds on

Learning-Based Animation of Clothing for Virtual Try-On 2019

Built upon by

Nothing yet.

Related work

PBNS: Physically Based Neural Simulation for Unsupervised Garment Pose Space Deformation 2021 / SIGGRAPH Asia
Neural Cloth Simulation 2022 / SIGGRAPH Asia
SwinGar: Spectrum-Inspired Neural Dynamic Deformation for Free-Swinging Garments 2024 / TVCG
A Pixel-Based Framework for Data-Driven Clothing 2020 / SCA

Keywords

self-supervised learning garment deformation dynamic garments cloth simulation physics-based loss optimization-based dynamics backward euler saint venant kirchhoff membrane strain energy bending energy collision penalty gated recurrent unit linear blend skinning smpl pose space deformation amass virtual try-on material model

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →