← ArchivePaper2022

Physics-Based Character Controllers Using Conditional VAEs

Jungdam Won, Deepak Gopinath, Jessica Hodgins

SIGGRAPHAcademic144 citesMotion Synthesis

Conditional VAE latent space over motion clips drives a physics controller, enabling goal-directed interactive locomotion from unstructured data.

Open the paper ↗Trace in the lineage graph

Abstract

High-quality motion capture datasets are now publicly available, and researchers have used them to create kinematics-based controllers that can generate plausible and diverse human motions without conditioning on specific goals (i.e., a task-agnostic generative model). In this paper, we present an algorithm to build such controllers for physically simulated characters having many degrees of freedom. Our physics-based controllers are learned by using conditional VAEs, which can perform a variety of behaviors that are similar to motions in the training dataset. The controllers are robust enough to generate more than a few minutes of motion without conditioning on specific goals and to allow many complex downstream tasks to be solved efficiently. To show the effectiveness of our method, we demonstrate controllers learned from several different motion capture databases and use them to solve a number of downstream tasks that are challenging to learn controllers that generate natural-looking motions from scratch. We also perform ablation studies to demonstrate the importance of the elements of the algorithm. Code and data for this paper are available at: https://github.com/facebookresearch/PhysicsVAE

How to read this

Category: Method: physics-based character controller from a conditional VAE
Contributions: An algorithm to build physics-based controllers for high-DOF simulated characters using conditional VAEs over motion capture data
A task-agnostic generative controller robust enough to produce minutes of motion without goal conditioning and to support diverse downstream tasks
Ablation studies isolating the importance of each algorithmic element, with code and data released
Context: Extends latent-variable motion modeling such as Motion VAEs (Ling et al. 2020) into the physically simulated, high-DOF control setting.Builds on: Character Controllers Using Motion VAEs
Correctness: Demonstrated on several motion capture databases and downstream tasks, so the behaviors it can express are inherited from the training motions; the ablations support the design choices but generalization beyond captured styles is not assumed.
Clarity: Accessible; a first pass conveys how the cVAE latent space drives the controller, a second pass covers the training and ablation details.
How to read it: Focus on the conditional VAE structure and how the latent conditions the physics controller; do a second pass on the ablations if you are designing a similar system.

Builds on

Character Controllers Using Motion VAEs 2020

Built upon by

Nothing yet.

Related work

Physical Based Motion Reconstruction From Videos Using Musculoskeletal Model 2024 / CASA
C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters 2023 / SIGGRAPH Asia
DReCon: Data-Driven Responsive Control of Physics-Based Characters 2019 / TOG
DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills 2018 / SIGGRAPH

Keywords

conditional vae physics-based character control motion capture behavior cloning deep reinforcement learning character animation motor primitives task-agnostic controller generative model locomotion control downstream task learning

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →