Model Predictive Control with a Visuomotor System for Physics-based Character Animation

How to read this

Category

Method: model predictive control with a visuomotor system for physics-based animation

Contributions

• An MPC framework that synthesizes coupled eye, head, and physics-based full-body motion while attending to important objects.
• A visuomotor system based on human visual perception and full-body dynamics with contacts, using partial, uncertain observations from a simulated visual sensor.
• A tractable formulation that approximates the resulting POMDP as a deterministic belief MDP and solves it with differential dynamic programming.

Context

Sits in the physics-based control lineage exemplified by Peng et al.'s DeepMimic, adding a perception-driven gaze-and-body coupling rather than relying on a priori gaze data.Builds on: DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills

Correctness

Demonstrated on tasks such as catching a thrown ball, walking on stepping stones, balancing after a push, and avoiding moving obstacles; results rely on the belief-MDP approximation and DDP finding a locally optimal policy, so behavior is local and task-guided by a reference skeletal motion.

Clarity

Mathematically dense; a first pass conveys the perceive-then-control loop, but the POMDP approximation and DDP need a careful second pass.

How to read it

Focus first on how visual attention feeds the controller and what the reference motion provides; a second pass on the belief-MDP and DDP formulation is needed to follow or reproduce the control.