← ArchivePaper2021
Dynamic Neural Garments
Neural network predicts pose-dependent dynamic wrinkle details for garments directly from skeletal motion without explicit cloth simulation.
Abstract
A vital task of the wider digital human effort is the creation of realistic garments on digital avatars, both in the form of characteristic fold patterns and wrinkles in static frames as well as richness of garment dynamics under avatars' motion. Existing workflow of modeling, simulation, and rendering closely replicates the physics behind real garments, but is tedious and requires repeating most of the workflow under changes to characters' motion, camera angle, or garment resizing. Although data-driven solutions exist, they either focus on static scenarios or only handle dynamics of tight garments. We present a solution that, at test time, takes in body joint motion to directly produce realistic dynamic garment image sequences. Specifically, given the target joint motion sequence of an avatar, we propose dynamic neural garments to synthesize plausible dynamic garment appearance from a desired viewpoint. Technically, our solution generates a coarse garment proxy sequence, learns deep dynamic features attached to this template, and neurally renders the features to produce appearance changes such as folds, wrinkles, and silhouettes. We demonstrate generalization behavior to both unseen motion and unseen camera views. Further, our network can be fine-tuned to adopt to new body shape and/or background images.
How to read this
- Category
- Method: neural rendering of pose-dependent dynamic garments
- Contributions
- Dynamic Neural Garments: at test time takes body joint motion and directly produces realistic dynamic garment image sequences from a desired viewpoint, without explicit cloth simulation
- A pipeline that generates a coarse garment proxy sequence, learns deep dynamic features attached to the template, and neurally renders them into folds, wrinkles, and silhouettes
- Demonstrated generalization to unseen motions
- Context
- Builds on learning-based clothing animation (e.g. virtual try-on), extending data-driven garments from static or tight-clothing cases to dynamic, looser garment appearance.Builds on: Learning-Based Animation of Clothing for Virtual Try-On
- Correctness
- Validated on garment image-sequence synthesis with generalization shown; note that it outputs rendered appearance from a viewpoint (image-space) rather than a simulated 3D cloth mesh, so it is best read as neural rendering, not physical simulation.
- Clarity
- Accessible at a first pass for the goal and pipeline stages; a second pass clarifies the feature-learning and neural-rendering steps.
- How to read it
- First pass for the coarse-proxy then learned-feature then neural-render pipeline and what is image-space vs. geometry; second pass on the dynamic feature representation if you work on neural garment appearance.
Builds on
Built upon by
Nothing yet.
Related work
- Motion Guided Deep Dynamic 3D Garments 2022 / SIGGRAPH Asia
- GarMatNet: A Learning-Based Method for Predicting 3D Garment Mesh with Parameterized Materials 2021 / MIG
- A Pixel-Based Framework for Data-Driven Clothing 2020 / SCA
- Dynamic Deformables: Implementation and Production Practicalities 2020 / SIGGRAPH Courses
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →