← ArchivePaper2020

Character Controllers Using Motion VAEs

Hung Yu Ling, Fabio Zinno, George Cheng, Michiel van de Panne

SIGGRAPHAcademic326 cites3 descendantsMotion Synthesis

Learns autoregressive conditional VAEs of human motion whose latent space serves as action space for deep reinforcement learning controllers achieving goal-directed locomotion.

Open the paper ↗Trace in the lineage graph

Abstract

A fundamental problem in computer animation is that of realizing purposeful and realistic human movement given a sufficiently-rich set of motion capture clips. We learn data-driven generative models of human movement using autoregressive conditional variational autoencoders, or Motion VAEs. The latent variables of the learned autoencoder define the action space for the movement and thereby govern its evolution over time. Planning or control algorithms can then use this action space to generate desired motions. In particular, we use deep reinforcement learning to learn controllers that achieve goal-directed movements. We demonstrate the effectiveness of the approach on multiple tasks. We further evaluate system-design choices and describe the current limitations of Motion VAEs.

How to read this

Category: Method: a learned generative motion model + RL controller
Contributions: Learns autoregressive conditional variational autoencoders (Motion VAEs) of human movement from mocap clips
Uses the VAE latent variables as the action space governing motion evolution over time
Trains deep reinforcement learning controllers in that action space for goal-directed locomotion, with ablations of system-design choices
Context: Builds on data-driven character control, succeeding phase-based approaches such as Holden et al.'s 'Phase-Functioned Neural Networks for Character Control' by replacing handcrafted structure with a learned latent action space.Builds on: Phase-Functioned Neural Networks for Character Control
Correctness: Assumes a learned autoregressive latent space is a well-behaved action space for RL and that the source clips are 'sufficiently rich'; demonstrated on multiple tasks with stated limitations, so quality depends on data coverage and the paper is candid about current limits.
Clarity: Fairly accessible for a learning paper; a first pass conveys the VAE-as-action-space idea, a second pass is needed for the autoregressive conditioning and RL setup.
How to read it: Focus on how the latent action space is defined and fed to RL, plus the limitations section; a second pass is worthwhile for the autoregressive conditioning details.

Builds on

Phase-Functioned Neural Networks for Character Control 2017

Built upon by

Related work

Interactive Character Control with Auto-Regressive Motion Diffusion Models 2024 / TOG
DeepPhase: Periodic Autoencoders for Learning Motion Phase Manifolds 2022 / SIGGRAPH
C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters 2023 / SIGGRAPH Asia
A Deep Learning Framework for Character Motion Synthesis and Editing 2016 / SIGGRAPH

Keywords

motion vae autoregressive conditional variational autoencoder character control motion synthesis deep reinforcement learning proximal policy optimization latent space mixture-of-experts decoder scheduled sampling posterior collapse locomotion motion capture kinematic motion model goal-directed control energy penalty reparameterization trick

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →