← ArchivePaper2023
Perpetual Humanoid Control for Real-time Simulated Avatars
Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, Weipeng Xu
Progressive multiplicative control policy (PMCP) scales physics imitation to 10,000 AMASS clips with real-time fault recovery.
Abstract
We present a physics-based humanoid controller that achieves high-fidelity motion imitation and fault-tolerant behavior in the presence of noisy input (e.g. pose estimates from video or generated from language) and unexpected falls. Our controller scales up to learning ten thousand motion clips without using any external stabilizing forces and learns to naturally recover from fail-state. Given reference motion, our controller can perpetually control simulated avatars without requiring resets. At its core, we propose the progressive multiplicative control policy (PMCP), which dynamically allocates new network capacity to learn harder and harder motion sequences. PMCP allows efficient scaling for learning from large-scale motion databases and adding new tasks, such as fail-state recovery, without catastrophic forgetting. We demonstrate the effectiveness of our controller by using it to imitate noisy poses from video-based pose estimators and language-based motion generators in a live and real-time multi-person avatar use case.
How to read this
- Category
- Method: physics-based humanoid control for simulated avatars
- Contributions
- A physics-based controller that imitates motion with high fidelity and recovers from fail-states without external stabilizing forces or resets
- Progressive Multiplicative Control Policy (PMCP) that allocates new network capacity to harder sequences and added tasks while avoiding catastrophic forgetting
- Scaling to ten thousand motion clips and real-time, multi-person control driven by noisy video- or language-based pose inputs
- Context
- Extends adversarial/imitation physics-based control such as AMP (Peng 2021) toward large-scale, perpetual, fault-tolerant tracking.Builds on: AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
- Correctness
- Demonstrated imitating large motion databases and noisy estimated poses in a simulator; results are in physics simulation, and tracking quality still depends on input pose noise.
- Clarity
- Moderately technical; a first pass conveys the PMCP scaling idea, a second pass is needed for the policy and training formulation.
- How to read it
- First pass for the PMCP capacity-allocation idea and fail-state recovery; second/third pass on the policy architecture and training if implementing physics-based imitation.
Built upon by
Related work
- CLoSD: Closing the Loop between Simulation and Diffusion for Multi-Task Character Control 2024 / arXiv
- Universal Humanoid Motion Representations for Physics-Based Control 2024 / ICCV
- SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation 2024 / SIGGRAPH
- AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control 2021 / SIGGRAPH
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →