← ArchivePaper2020

A Pixel-Based Framework for Data-Driven Clothing

Ning Jin, Yilin Zhu, Zhenglin Geng, Ron Fedkiw

Represents garment deformations as UV-space images enabling convolutional neural networks to predict pose-dependent clothing deformation efficiently.

Open the paper ↗Trace in the lineage graph

Abstract

We propose a novel approach to learning cloth deformation as a function of body pose, recasting the graph‐like triangle mesh data structure into image‐based data in order to leverage popular and well‐developed convolutional neural networks (CNNs) in a two‐dimensional Euclidean domain. Then, a three‐dimensional animation of clothing is equivalent to a sequence of two‐dimensional RGB images driven/choreographed by time dependent joint angles. In order to reduce nonlinearity demands on the neural network, we utilize procedural skinning of the body surface to capture much of the rotation/deformation so that the RGB images only contain textures of displacement offsets from skin to clothing. Notably, we illustrate that our approach does not require accurate unclothed body shapes or robust skinning techniques. Additionally, we discuss how standard image based techniques such as image partitioning for higher resolution can readily be incorporated into our framework.

How to read this

Category: Method: data-driven, pose-dependent clothing deformation
Contributions: Recasting triangle-mesh garment deformation as UV-space RGB images so 2D CNNs can be applied directly
Procedural body skinning that absorbs most rotation so the images only encode skin-to-clothing displacement offsets, reducing network nonlinearity
A framework tolerant of inaccurate unclothed body shapes and skinning, with standard image techniques (e.g. partitioning) reused for higher resolution
Context: Relates to learning-based clothing animation (referenced Learning-Based Animation of Clothing for Virtual Try-On, Santesteban 2019), reframing the problem in an image-based Euclidean domain.Builds on: Learning-Based Animation of Clothing for Virtual Try-On
Correctness: Demonstrated as a clothing animation driven by joint angles; the pixel/UV reformulation assumes deformation maps well to a 2D image domain, so seams, UV distortion, and topology effects are inherent considerations the abstract addresses only partially via partitioning.
Clarity: Clearly motivated and intuitive (mesh-to-image analogy); a first pass conveys the representation, a second pass is needed for the network and displacement-encoding details.
How to read it: Focus first on why the UV-image plus skinned-offset representation tames nonlinearity; second pass on the image partitioning and resolution handling if you plan to extend it to new garments or higher detail.

Builds on

Learning-Based Animation of Clothing for Virtual Try-On 2019

Built upon by

Nothing yet.

Related work

Stable Spaces for Real-time Clothing 2010 / SIGGRAPH
PBNS: Physically Based Neural Simulation for Unsupervised Garment Pose Space Deformation 2021 / SIGGRAPH Asia
Real-Time Hair Simulation with Neural Interpolation 2022 / TVCG
SNUG: Self-Supervised Neural Dynamic Garments 2022 / CVPR

Keywords

data-driven clothing cloth pixels convolutional neural network displacement maps pattern space body skinning linear blend skinning uv texture coordinates barycentric embedding generative adversarial network cloth simulation scape smpl pca wrinkles convolutional decoder cage and patch blendshapes

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →