← ArchivePaper2023

CALM: Conditional Adversarial Latent Models for Directable Virtual Characters

Chen Tessler, Yoni Kasten, Yunrong Guo, Shie Mannor, Gal Chechik, Xue Bin Peng

SIGGRAPHAcademic150 cites1 descendantMotion Synthesis

Jointly learns a physics-based control policy and motion encoder for diverse, directable character behavior from mocap via imitation learning.

Open the paper ↗Trace in the lineage graph

Abstract

In this work, we present Conditional Adversarial Latent Models (CALM), an approach for generating diverse and directable behaviors for user-controlled interactive virtual characters. Using imitation learning, CALM learns a representation of movement that captures the complexity and diversity of human motion, and enables direct control over character movements. The approach jointly learns a control policy and a motion encoder that reconstructs key characteristics of a given motion without merely replicating it. The results show that CALM learns a semantic motion representation, enabling control over the generated motions and style-conditioning for higher-level task training. Once trained, the character can be controlled using intuitive interfaces, akin to those found in video games.

How to read this

Category: Method: conditional adversarial latent model for directable physics-based characters
Contributions: Generates diverse, directable behaviors for user-controlled interactive characters via imitation learning
Jointly learns a control policy and a motion encoder that reconstructs key motion characteristics without merely replicating them
Yields a semantic motion representation enabling direct control and style-conditioning for higher-level task training
Context: Part of the adversarial skill embedding lineage, building on ASE (Peng et al. 2022) toward game-style directable control.Builds on: ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters
Correctness: Demonstrated on mocap-driven character control with game-like interfaces; the claim of a semantic representation depends on the encoder capturing motion characteristics rather than overfitting, and directability quality is shown qualitatively.
Clarity: Accessible, framed around familiar game-control intuitions; a first pass conveys the idea, a second pass clarifies the joint policy and encoder objective.
How to read it: Focus on how the encoder and policy are trained jointly and what makes the latent space directable; a second pass is worth it to see how style-conditioning feeds higher-level tasks.

Builds on

ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters 2022

Built upon by

MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting 2024

Related work

ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters 2022 / SIGGRAPH
C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters 2023 / SIGGRAPH Asia
PADL: Language-Directed Physics-Based Character Control 2022 / SIGGRAPH Asia
SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation 2024 / SIGGRAPH

Keywords

conditional adversarial latent models physics-based character control imitation learning adversarial imitation learning motion encoder latent representation reinforcement learning proximal policy optimization conditional discriminator motion capture hierarchical reinforcement learning high-level policy low-level policy precision training finite state machine skill embedding isaac gym directable characters mode collapse

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →