← ArchivePaper2021
PFPN: Continuous Control of Physically Simulated Characters using Particle Filtering Policy Network
Particle filtering policy network enabling continuous control of physically simulated characters with improved robustness and diversity.
Abstract
Data-driven methods for physics-based character control using reinforcement learning have been successfully applied to generate high-quality motions. However, existing approaches typically rely on Gaussian distributions to represent the action policy, which can prematurely commit to suboptimal actions when solving high-dimensional continuous control problems for highly-articulated characters. In this paper, to improve the learning performance of physics-based character controllers, we propose a framework that considers a particle-based action policy as a substitute for Gaussian policies. We exploit particle filtering to dynamically explore and discretize the action space, and track the posterior policy represented as a mixture distribution. The resulting policy can replace the unimodal Gaussian policy which has been the staple for character control problems, without changing the underlying model architecture of the reinforcement learning algorithm used to perform policy optimization. We demonstrate the applicability of our approach on various motion capture imitation tasks. Baselines using our particle-based policies achieve better imitation performance and speed of convergence as compared to corresponding implementations using Gaussians, and are more robust to external perturbations during character control. Related code is available at: https://motion-lab.github.io/PFPN.
How to read this
- Category
- Method: a reinforcement-learning policy representation for physics-based character control
- Contributions
- Proposes a particle-based action policy (PFPN) as a drop-in substitute for the usual Gaussian policy in RL character control
- Uses particle filtering to dynamically explore and discretize the action space and track a mixture posterior policy
- Reports improved imitation performance and convergence speed on motion-capture imitation tasks without changing the underlying RL architecture
- Context
- Builds on example-guided physics-based RL control in the DeepMimic lineage (Peng et al.), addressing the limitation that unimodal Gaussian policies can commit prematurely in high-dimensional articulated control.Builds on: DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
- Correctness
- Demonstrated on motion-capture imitation tasks as a policy substitution, so gains are shown within that setting; the particle representation adds machinery and the benefit outside imitation or at different action dimensionalities is not something to assume from the abstract.
- Clarity
- Moderately technical; a first pass conveys the Gaussian-versus-particle-policy motivation, a second pass is needed for the particle-filtering update and mixture-policy details.
- How to read it
- First pass to grasp why a multimodal particle policy helps articulated control; do a second pass on the particle-filter mechanics only if you work in RL motion imitation and want to swap the policy class.
Builds on
Built upon by
Nothing yet.
Related work
- Composite Motion Learning with Task Control 2023 / SIGGRAPH
- DReCon: Data-Driven Responsive Control of Physics-Based Characters 2019 / TOG
- UniCon: Universal Neural Controller for Physics-Based Character Motion 2020 / arXiv
- CALM: Conditional Adversarial Latent Models for Directable Virtual Characters 2023 / SIGGRAPH
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →