← ArchivePaper2023

ACE: Adversarial Correspondence Embedding for Cross Morphology Motion Retargeting from Human to Nonhuman Characters

Tianyu Li, Jungdam Won, Alexander Clegg, Jeonghwan Kim, Akshara Rai, Sehoon Ha

SIGGRAPH AsiaAcademic32 citesRetargeting

Adversarial correspondence learning retargets human motion to nonhuman characters with different body structure and proportions.

Open the paper ↗Trace in the lineage graph

Abstract

Motion retargeting is a promising approach for generating natural and compelling animations for nonhuman characters. However, it is challenging to translate human movements into semantically equivalent motions for target characters with different morphologies due to the ambiguous nature of the problem. This work presents a novel learning-based motion retargeting framework, Adversarial Correspondence Embedding (ACE), to retarget human motions onto target characters with different body dimensions and structures. Our framework is designed to produce natural and feasible character motions by leveraging generative-adversarial networks (GANs) while preserving high-level motion semantics by introducing an additional feature loss. In addition, we pretrain a character motion prior that can be controlled in a latent embedding space and seek to establish a compact correspondence. We demonstrate that the proposed framework can produce retargeted motions for three different characters, a quadrupedal robot with a manipulator, a crab character, and a wheeled manipulator. We further validate the design choices of our framework by conducting baseline comparisons and a user study. We also showcase sim-to-real transfer of the retargeted motions by transferring them to a real Spot robot.

How to read this

Category: Method: a cross-morphology motion retargeting framework
Contributions: Adversarial Correspondence Embedding (ACE) retargeting human motion onto characters with different body structure and proportions
A GAN-based generator with an added feature loss to preserve high-level motion semantics
A pretrained, latent-controllable character motion prior used to establish compact correspondence
Context: Builds on deep retargeting work such as Aberman's Skeleton-Aware Networks, extending beyond skeleton matching to non-human morphologies via adversarial correspondence.Builds on: Skeleton-Aware Networks for Deep Motion Retargeting
Correctness: Validated on three target characters (a quadrupedal robot with manipulator, a crab, and a wheeled manipulator) plus baseline comparisons and a user study; the cross-morphology mapping is inherently ambiguous, so results depend on the chosen characters and perceptual judging.
Clarity: Moderately accessible; a first pass conveys the adversarial-plus-semantic-loss idea, a second pass is needed for the embedding and prior training.
How to read it: Focus on how correspondence is learned adversarially and what the feature loss preserves; a second pass on the motion prior is worth it if you target a new character morphology.

Builds on

Skeleton-Aware Networks for Deep Motion Retargeting 2020

Built upon by

Nothing yet.

Related work

Skeleton-Aware Networks for Deep Motion Retargeting 2020 / SIGGRAPH
WalkTheDog: Cross-Morphology Motion Alignment via Phase Manifolds 2024 / SIGGRAPH
Normalized Euclidean Distance Matrices for Human Motion Retargeting 2017 / MIG
Dog Code: Human to Quadruped Embodiment Using Shared Codebooks 2024 / MIG

Keywords

motion retargeting cross-morphology retargeting adversarial correspondence embedding generative adversarial networks motion prior latent embedding space variational autoencoder mode-adaptive neural network feature loss end-effector correspondence quadrupedal robot sim-to-real transfer nonhuman characters frechet inception distance unpaired translation discriminator gradient penalty

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →