← ArchivePaper2023

Invertible Neural Skinning

Yash Kant, Aliaksandr Siarohin, Riza Alp Guler, Menglei Chai, Jian Ren, Sergey Tulyakov, Igor Gilitschenski

CVPRAcademic3 citesSkinning ML Deformation

Combines a pose-conditioned invertible network with differentiable LBS to animate implicit surfaces while preserving surface correspondences, outperforming state-of-the-art reposing methods.

Open the paper ↗Trace in the lineage graph

Abstract

Building animatable and editable models of clothed humans from raw 3D scans and poses is a challenging problem. Existing reposing methods suffer from the limited expressiveness of Linear Blend Skinning (LBS), require costly mesh extraction to generate each new pose, and typically do not preserve surface correspondences across different poses. In this work, we introduce Invertible Neural Skinning (INS) to address these shortcomings. To maintain correspondences, we propose a Pose-conditioned Invertible Network (PIN) architecture, which extends the LBS process by learning additional pose-varying deformations. Next, we combine PIN with a differentiable LBS module to build an expressive and end-to-end Invertible Neural Skinning (INS) pipeline. We demonstrate the strong performance of our method by outperforming the state-of-the-art reposing techniques on clothed humans and preserving surface correspondences, while being an order of magnitude faster. We also perform an ablation study, which shows the usefulness of our pose-conditioning formulation, and our qualitative results display that INS can rectify artefacts introduced by LBS well.

How to read this

Category: Method: an invertible neural skinning pipeline for implicit surfaces
Contributions: A Pose-conditioned Invertible Network (PIN) that extends LBS with learned pose-varying deformations while maintaining surface correspondences
An end-to-end Invertible Neural Skinning pipeline combining PIN with a differentiable LBS module, avoiding costly per-pose mesh extraction
Reported state-of-the-art reposing quality on clothed humans at roughly an order of magnitude faster, with an ablation on the pose-conditioning
Context: Builds on differentiable forward skinning of neural implicit shapes (SNARF) and on classic Linear Blend Skinning, targeting LBS's limited expressiveness and lack of correspondence.Builds on: SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes
Correctness: Validated on clothed-human reposing from 3D scans and poses; gains in speed and correspondence rest on the invertibility of PIN, so generalization to poses far from training and to very loose garments is a reader caution.
Clarity: Moderately technical; a first pass conveys the invertible-skinning idea, a second pass is needed for the network invertibility and its coupling with LBS.
How to read it: First pass for why correspondence and mesh-extraction cost matter; second pass on the PIN architecture and the differentiable-LBS coupling if you intend to reimplement.

Builds on

SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes 2021

Built upon by

Nothing yet.

Related work

Keywords

invertible neural skinning linear blend skinning pose-conditioned invertible network invertible neural networks coupling layers occupancy network clothed humans reposing canonical space differentiable lbs surface correspondences snarf broyden's method siren cape dataset dfaust pose-varying deformation normalizing flows mesh extraction

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →