← ArchivePaper2020
Skeleton-Aware Networks for Deep Motion Retargeting
Kfir Aberman, Peizhuo Li, Dani Lischinski, Olga Sorkine-Hornung, Daniel Cohen-Or, Baoquan Chen
Skeleton-aware graph network for motion retargeting that adapts motion to diverse skeleton proportions while preserving stylistic details.
Abstract
We introduce a novel deep learning framework for data-driven motion retargeting between skeletons, which may have different structure, yet corresponding to homeomorphic graphs. Importantly, our approach learns how to retarget without requiring any explicit pairing between the motions in the training set. We leverage the fact that different homeomorphic skeletons may be reduced to a common primal skeleton by a sequence of edge merging operations, which we refer to as skeletal pooling. Thus, our main technical contribution is the introduction of novel differentiable convolution, pooling, and unpooling operators. These operators are skeleton-aware, meaning that they explicitly account for the skeleton's hierarchical structure and joint adjacency, and together they serve to transform the original motion into a collection of deep temporal features associated with the joints of the primal skeleton. In other words, our operators form the building blocks of a new deep motion processing framework that embeds the motion into a common latent space, shared by a collection of homeomorphic skeletons. Thus, retargeting can be achieved simply by encoding to, and decoding from this latent space. Our experiments show the effectiveness of our framework for motion retargeting, as well as motion processing in general, compared to existing approaches.
How to read this
- Category
- Method: deep motion retargeting via skeleton-aware networks
- Contributions
- A deep framework that retargets motion between skeletons of different structure (homeomorphic graphs) without requiring paired training motions
- Novel skeleton-aware differentiable convolution, pooling, and unpooling operators that respect joint hierarchy and adjacency
- A shared latent space reached by skeletal pooling to a common primal skeleton, so retargeting becomes encode-then-decode
- Context
- Builds on the authors' earlier character-agnostic 2D motion retargeting work, moving it into a graph-based, skeleton-aware deep operator framework for 3D.Builds on: Learning Character-Agnostic Motion for Motion Retargeting in 2D
- Correctness
- Trained without explicit motion pairing and relying on the assumption that target skeletons reduce to a common primal skeleton via edge merging; a reader should keep in mind this homeomorphism requirement limits applicability to topologically compatible skeletons.
- Clarity
- Moderately technical; a first pass conveys the pooling-to-primal-skeleton idea, a second pass is needed to understand the operator definitions.
- How to read it
- First pass on the skeletal pooling concept and the encode/decode latent-space picture; second pass on the convolution/pooling/unpooling operators if you plan to implement or extend them.
Built upon by
- Motion Retargetting based on Dilated Convolutions and Skeleton-Specific Loss Functions 2020
- Unpaired Motion Style Transfer from Video to Animation 2020
- Contact-Aware Retargeting of Skinned Motion 2021
- ACE: Adversarial Correspondence Embedding for Cross Morphology Motion Retargeting from Human to Nonhuman Characters 2023
- SAME: Skeleton-Agnostic Motion Embedding for Character Animation 2023
- WalkTheDog: Cross-Morphology Motion Alignment via Phase Manifolds 2024
Related work
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →