← ArchivePaper2018
Physics-based Motion Capture Imitation with Deep Reinforcement Learning
Nuttapong Chentanez, Matthias Muller, Miles Macklin, Viktor Makoviychuk, Stefan Jeschke
Physics-based character controller that imitates motion capture reference with deep reinforcement learning, producing physically plausible results.
Abstract
We introduce a deep reinforcement learning method that learns to control articulated humanoid bodies to imitate given target motions closely when simulated in a physics simulator. The target motion, which may not have been seen by the agent and can be noisy, is supplied at runtime. Our method can recover balance from moderate external disturbances and keep imitating the target motion. When subjected to large disturbances that cause the humanoid to fall down, our method can control the character to get up and recover to track the motion. Our method is trained to imitate the mocap clips from the CMU motion capture database and a number of other publicly available databases. We use a state-of-the-art deep reinforcement learning algorithm to learn to dynamically control the gain of PD controllers, whose target angles are derived from the mocap clip and to apply corrective torques with the goal of imitating the provided motion clip as closely as possible. Both the simulation and the learning algorithms are parallelized and run on the GPU. We demonstrate that the proposed method can control the character to imitate a wide variety of motions such as running, walking, dancing, jumping, kicking, punching, standing up, and so on.
How to read this
- Category
- Method: a physics-based motion-imitation controller via deep RL
- Contributions
- A deep RL method that controls articulated humanoids to closely imitate target motions in a physics simulator
- Handles runtime target motions that may be unseen or noisy, and recovers balance after disturbances including getting up from falls
- Learns to dynamically tune PD-controller gains and apply corrective torques, with GPU-parallelized simulation and training
- Context
- Builds on example-guided physics-based RL controllers (DeepMimic), trained to imitate clips from the CMU and other public mocap databases.Builds on: DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
- Correctness
- Demonstrated across a wide variety of motions (running, walking, dancing, jumping, etc.) and disturbance recovery; a reader should keep in mind that imitation fidelity and recovery depend on the simulator, reward design, and the coverage of the training clips.
- Clarity
- Moderately technical; a first pass conveys the imitate-with-RL idea and the runtime-target feature, a second pass covers the state, reward, and gain-control details.
- How to read it
- First pass to grasp the runtime-supplied target and the PD-gain control; do a second pass on the reward and training pipeline, especially to contrast its design choices with DeepMimic.
Builds on
Built upon by
Nothing yet.
Related work
- SFV: Reinforcement Learning of Physical Skills from Video 2018 / SIGGRAPH Asia
- DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills 2018 / SIGGRAPH
- C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters 2023 / SIGGRAPH Asia
- DReCon: Data-Driven Responsive Control of Physics-Based Characters 2019 / TOG
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →