← ArchivePaper2024

Interactive Character Control with Auto-Regressive Motion Diffusion Models

Yi Shi, Jingbo Wang, Xuekun Jiang, Bingkun Lin, Bo Dai, Xue Bin Peng

Auto-regressive diffusion model generates physically plausible real-time character motion responding interactively to user control signals.

Open the paper ↗Trace in the lineage graph

Abstract

Real-time character control is an essential component for interactive experiences, with a broad range of applications, including physics simulations, video games, and virtual reality. The success of diffusion models for image synthesis has led to the use of these models for motion synthesis. However, the majority of these motion diffusion models are primarily designed for offline applications, where space-time models are used to synthesize an entire sequence of frames simultaneously with a pre-specified length. To enable real-time motion synthesis with diffusion model that allows time-varying controls, we propose A-MDM (Auto-regressive Motion Diffusion Model). Our conditional diffusion model takes an initial pose as input, and auto-regressively generates successive motion frames conditioned on the previous frame. Despite its streamlined network architecture, which uses simple MLPs, our framework is capable of generating diverse, long-horizon, and high-fidelity motion sequences. Furthermore, we introduce a suite of techniques for incorporating interactive controls into A-MDM, such as task-oriented sampling, in-painting, and hierarchical reinforcement learning (See Figure 1). These techniques enable a pre-trained A-MDM to be efficiently adapted for a variety of new downstream tasks.

How to read this

Category: Method: real-time motion diffusion for character control
Contributions: A-MDM, an auto-regressive conditional motion diffusion model that takes an initial pose and generates successive frames conditioned on the previous frame, enabling real-time, time-varying control.
A streamlined architecture using simple MLPs that still produces diverse, long-horizon, high-fidelity motion.
A suite of interactive-control techniques: task-oriented sampling, in-painting, and hierarchical reinforcement learning.
Context: Adapts motion diffusion (Tevet et al., 'Human Motion Diffusion Model') from offline whole-sequence synthesis to an auto-regressive, frame-by-frame formulation for interactive control.Builds on: Human Motion Diffusion Model
Correctness: Built on the assumption that frame-conditioned auto-regression preserves quality while gaining real-time controllability; long-horizon stability and control fidelity rest on the added sampling/in-painting/RL techniques, so a reader should watch for drift and the cost of the control machinery.
Clarity: Accessible if familiar with diffusion-based motion; a first pass conveys the auto-regressive shift and control toolkit, a second pass for the conditioning and RL details.
How to read it: First pass for the offline-to-auto-regressive reframing and the three control techniques; second pass on the sampling/in-painting/hierarchical-RL mechanics if building interactive control.

Builds on

Human Motion Diffusion Model 2022

Built upon by

Nothing yet.

Related work

Character Controllers Using Motion VAEs 2020 / SIGGRAPH
SKEL-Betweener: a Neural Motion Rig for Interactive Motion Authoring 2024 / SIGGRAPH Asia
PDP: Physics-Based Character Animation via Diffusion Policy 2024 / SIGGRAPH Asia
CLoSD: Closing the Loop between Simulation and Diffusion for Multi-Task Character Control 2024 / arXiv

Keywords

auto-regressive motion diffusion model a-mdm motion synthesis diffusion model ddpm reinforcement learning hierarchical control proximal policy optimization scheduled sampling motion in-painting keyframe inbetweening amass multi-layer perceptron real-time character control motion vae 6d rotation representation

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →