← ArchivePaper2023
Synthesizing Physical Character-Scene Interactions
Mohamed Hassan, Yunrong Guo, Tingwu Wang, Michael Black, Sanja Fidler, Xue Bin Peng
RL with adversarial imitation trains physically simulated characters to perform carrying, sitting, and lying down from unstructured mocap data.
Abstract
Movement is how people interact with and affect their environment. For realistic character animation, it is necessary to synthesize such interactions between virtual characters and their surroundings. Despite recent progress in character animation using machine learning, most systems focus on controlling an agent’s movements in fairly simple and homogeneous environments, with limited interactions with other objects. Furthermore, many previous approaches that synthesize human-scene interactions require significant manual labeling of the training data. In contrast, we present a system that uses adversarial imitation learning and reinforcement learning to train physically-simulated characters that perform scene interaction tasks in a natural and life-like manner. Our method learns scene interaction behaviors from large unstructured motion datasets, without manual annotation of the motion data. These scene interactions are learned using an adversarial discriminator that evaluates the realism of a motion within the context of a scene. The key novelty involves conditioning both the discriminator and the policy networks on scene context. We demonstrate the effectiveness of our approach through three challenging scene interaction tasks: carrying, sitting, and lying down, which require coordination of a character’s movements in relation to objects in the environment.
How to read this
- Category
- Method: physics-based character-scene interaction via RL
- Contributions
- A system that synthesizes physically simulated character-scene interactions such as carrying, sitting, and lying down
- Learns these behaviors from large unstructured motion datasets without manual annotation
- Conditions both the adversarial discriminator and the policy on scene context to judge motion realism within a scene
- Context
- Combines scene-conditioned control in the spirit of the Neural State Machine (Starke et al.) with adversarial imitation from AMP (Peng et al.).Builds on: Neural State Machine for Character-Scene Interactions · AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
- Correctness
- The key idea is that a scene-conditioned discriminator can supply a realism signal without labels; demonstrated on interaction tasks like sitting, carrying, and lying down, with generality across novel objects and scene layouts being the natural thing to probe.
- Clarity
- Accessible at the conceptual level; a first pass conveys the scene-conditioned adversarial-imitation idea, a second pass clarifies the policy and discriminator design.
- How to read it
- Focus on how scene context enters the discriminator and policy and why that removes manual labeling; a second pass on the RL setup and reward structure helps if reproducing or extending.
Builds on
Built upon by
Nothing yet.
Related work
- AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control 2021 / SIGGRAPH
- ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters 2022 / SIGGRAPH
- DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills 2018 / SIGGRAPH
- Neural State Machine for Character-Scene Interactions 2019 / SIGGRAPH Asia
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →