← ArchivePaper2016
Terrain-Adaptive Locomotion Skills Using Deep Reinforcement Learning
Deep reinforcement learning for terrain-adaptive character locomotion that generalizes to novel obstacles without motion capture references.
Abstract
Reinforcement learning offers a promising methodology for developing skills for simulated characters, but typically requires working with sparse hand-crafted features. Building on recent progress in deep reinforcement learning (DeepRL), we introduce a mixture of actor-critic experts (MACE) approach that learns terrain-adaptive dynamic locomotion skills using high-dimensional state and terrain descriptions as input, and parameterized leaps or steps as output actions. MACE learns more quickly than a single actor-critic approach and results in actor-critic experts that exhibit specialization. Additional elements of our solution that contribute towards efficient learning include Boltzmann exploration and the use of initial actor biases to encourage specialization. Results are demonstrated for multiple planar characters and terrain classes.
How to read this
- Category
- Method: deep reinforcement learning for terrain-adaptive locomotion control
- Contributions
- A mixture of actor-critic experts (MACE) approach that learns terrain-adaptive dynamic locomotion from high-dimensional state and terrain descriptions, outputting parameterized leaps or steps
- Shows MACE learns faster than a single actor-critic and yields specialized experts
- Uses Boltzmann exploration and initial actor biases to encourage specialization and efficient learning, demonstrated across multiple planar characters and terrain classes
- Context
- Extends physics-based locomotion control (such as SIMBICON) and the evolved-controller tradition (Sims, Evolving Virtual Creatures) using deep reinforcement learning to remove hand-crafted feature dependence.Builds on: SIMBICON: Simple Biped Locomotion Control · Evolving Virtual Creatures
- Correctness
- Results are demonstrated on planar (2D) simulated characters over several terrain classes, so generalization to 3D characters and the realism of motion remain to be judged; the action space is restricted to parameterized leaps or steps.
- Clarity
- Conceptually accessible if you know actor-critic RL; a first pass conveys the MACE idea, with training details in later passes.
- How to read it
- First pass for the MACE experts-and-specialization idea and the input/output design; second pass on exploration, biasing, and the planar-only scope before assuming it transfers to your setting.
Builds on
Related work
- Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter? 2017 / SCA
- DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills 2018 / SIGGRAPH
- Physics-based Motion Capture Imitation with Deep Reinforcement Learning 2018 / MIG
- SFV: Reinforcement Learning of Physical Skills from Video 2018 / SIGGRAPH Asia
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →