← ArchivePaper2022

Human Motion Diffusion Model

Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, Amit H. Bermano

arXivAcademic1398 cites9 descendantsMotion Synthesis

Transformer-based diffusion model for text-to-motion and action-to-motion synthesis achieving state-of-the-art while predicting samples not noise.

Open the paper ↗Trace in the lineage graph

Abstract

Natural and expressive human motion generation is the holy grail of computer animation. It is a challenging task, due to the diversity of possible motion, human perceptual sensitivity to it, and the difficulty of accurately describing it. Therefore, current generative solutions are either low-quality or limited in expressiveness. Diffusion models, which have already shown remarkable generative capabilities in other domains, are promising candidates for human motion due to their many-to-many nature, but they tend to be resource hungry and hard to control. In this paper, we introduce Motion Diffusion Model (MDM), a carefully adapted classifier-free diffusion-based generative model for the human motion domain. MDM is transformer-based, combining insights from motion generation literature. A notable design-choice is the prediction of the sample, rather than the noise, in each diffusion step. This facilitates the use of established geometric losses on the locations and velocities of the motion, such as the foot contact loss. As we demonstrate, MDM is a generic approach, enabling different modes of conditioning, and different generation tasks. We show that our model is trained with lightweight resources and yet achieves state-of-the-art results on leading benchmarks for text-to-motion and action-to-motion. https://guytevet.github.io/mdm-page/ .

How to read this

Category: Method: a generative model for human motion
Contributions: MDM, a transformer-based classifier-free diffusion model adapted to the human-motion domain
Predicting the sample rather than the noise each step, enabling geometric losses such as a foot-contact loss
A generic approach supporting multiple conditioning modes and tasks (text-to-motion, action-to-motion)
Context: Combines diffusion-model advances from other domains with motion-generation literature and text-to-motion datasets (related to Guo et al.'s HumanML3D, 2022).Builds on: Generating Diverse and Natural 3D Human Motions from Text
Correctness: The central design choice (predict the sample, not the noise) is motivated by enabling location/velocity losses; the abstract claims state-of-the-art and light training but a reader should withhold judgment on numbers until the experiments section and watch how human perceptual sensitivity to motion is evaluated.
Clarity: Conceptually accessible if you know diffusion basics; a first pass conveys the architecture and key design choice, a second pass for the loss formulation and conditioning.
How to read it: First pass on the sample-prediction choice and conditioning modes; do a focused second pass on the geometric losses (especially foot contact) and the diffusion mechanics if you plan to build on it.

Builds on

Generating Diverse and Natural 3D Human Motions from Text 2022

Built upon by

Related work

Executing Your Commands via Motion Diffusion in Latent Space 2023 / CVPR
MotionCLIP: Exposing Human Motion Generation to CLIP Space 2022 / CVPR
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model 2022 / arXiv
CLoSD: Closing the Loop between Simulation and Diffusion for Multi-Task Character Control 2024 / arXiv

Keywords

motion diffusion model human motion generation text-to-motion action-to-motion classifier-free guidance diffusion model transformer encoder geometric losses foot contact loss forward kinematics motion in-betweening motion editing inpainting clip humanml3d amass kit dataset

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →