← ArchivePaper2024
MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting
Chen Tessler, Yunrong Guo, Ofir Nabati, Gal Chechik, Xue Bin Peng
Frames physics-based character control as motion inpainting so one model handles keyframes, objects, text, and sensor constraints.
Abstract
Crafting a single, versatile physics-based controller that can breathe life into interactive characters across a wide spectrum of scenarios represents an exciting frontier in character animation. An ideal controller should support diverse control modalities, such as sparse target keyframes, text instructions, and scene information. While previous works have proposed physically simulated, scene-aware control models, these systems have predominantly focused on developing controllers that each specializes in a narrow set of tasks and control modalities. This work presents MaskedMimic, a novel approach that formulates physics-based character control as a general motion inpainting problem. Our key insight is to train a single unified model to synthesize motions from partial (masked) motion descriptions, such as masked keyframes, objects, text descriptions, or any combination thereof. This is achieved by leveraging motion tracking data and designing a scalable training method that can effectively utilize diverse motion descriptions to produce coherent animations. Through this process, our approach learns a physics-based controller that provides an intuitive control interface without requiring tedious reward engineering for all behaviors of interest. The resulting controller supports a wide range of control modalities and enables seamless transitions between disparate tasks.
How to read this
- Category
- Method: a unified physics-based character controller
- Contributions
- MaskedMimic, formulating physics-based control as motion inpainting from partial (masked) descriptions
- A single model handling masked keyframes, objects, text, and combinations as control modalities
- A scalable training method using motion-tracking data to produce coherent controllable animation
- Context
- Extends adversarial and latent-space physics-based control (e.g. Tessler et al.'s CALM) toward one general controller spanning many control modalities.Builds on: CALM: Conditional Adversarial Latent Models for Directable Virtual Characters
- Correctness
- The inpainting framing assumes diverse partial descriptions can be unified under masking; it is demonstrated on physically simulated control across modalities, so a reader should note which task mixes and constraint types are actually shown versus claimed generality.
- Clarity
- Accessible at a conceptual level; a first pass conveys the masking idea, a second pass for the masking scheme and training pipeline.
- How to read it
- Read the masking formulation and how modalities are encoded first; a second pass pays off for the training method and the breadth of demonstrated tasks.
Built upon by
Nothing yet.
Related work
- SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation 2024 / SIGGRAPH
- CLoSD: Closing the Loop between Simulation and Diffusion for Multi-Task Character Control 2024 / arXiv
- MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations 2024 / TOG
- Generating Diverse and Natural 3D Human Motions from Text 2022 / CVPR
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →