← ArchivePaper2021
Action-Conditioned 3D Human Motion Synthesis with Transformer VAE
ACTOR uses a Transformer VAE to synthesize variable-length SMPL motion sequences conditioned on action category labels.
Abstract
We tackle the problem of action-conditioned generation of realistic and diverse human motion sequences. In contrast to methods that complete, or extend, motion sequences, this task does not require an initial pose or sequence. Here we learn an action-aware latent representation for human motions by training a generative variational autoencoder (VAE). By sampling from this latent space and querying a certain duration through a series of positional encodings, we synthesize variable-length motion sequences conditioned on a categorical action. Specifically, we design a Transformer-based architecture, ACTOR, for encoding and decoding a sequence of parametric SMPL human body models estimated from action recognition datasets. We evaluate our approach on the NTU RGB+D, HumanAct12 and UESTC datasets and show improvements over the state of the art. Furthermore, we present two use cases: improving action recognition through adding our synthesized data to training, and motion denoising. Code and models are available on our project page [53].
How to read this
- Category
- Method: action-conditioned generative model for human motion
- Contributions
- A Transformer VAE (ACTOR) that learns an action-aware latent space and decodes variable-length SMPL motion sequences from a categorical action label
- Sampling plus positional-encoding queries to control sequence duration without needing an initial pose or seed sequence
- Two demonstrated use cases: augmenting action-recognition training data and motion denoising
- Context
- Generative human-motion synthesis on parametric SMPL bodies, combining VAE latent learning with a Transformer sequence model and evaluated against prior action-recognition motion datasets.
- Correctness
- Validated on NTU RGB+D, HumanAct12 and UESTC with reported improvements over prior work; keep in mind it conditions on coarse action categories (not fine-grained control) and operates on SMPL bodies estimated from action-recognition data, so input-estimate quality bounds the result.
- Clarity
- Accessible at a first pass for the idea; a second pass pays off for the VAE objective and the positional-encoding duration mechanism.
- How to read it
- First pass for the action-conditioned VAE concept and the no-seed-pose framing; second pass on the Transformer encoder/decoder design and loss if you intend to reimplement or extend conditioning.
Builds on
Nothing in the archive, this is a starting point.
Related work
- TEMOS: Generating Diverse Human Motions from Textual Descriptions 2022 / CVPR
- MotionCLIP: Exposing Human Motion Generation to CLIP Space 2022 / CVPR
- Executing Your Commands via Motion Diffusion in Latent Space 2023 / CVPR
- SKEL-Betweener: a Neural Motion Rig for Interactive Motion Authoring 2024 / SIGGRAPH Asia
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →