← ArchivePaper2016
A Deep Learning Framework for Character Motion Synthesis and Editing
Convolutional autoencoder for human motion that enables synthesis and editing of character motion sequences in a compact latent space.
Abstract
We present a framework to synthesize character movements based on high level parameters, such that the produced movements respect the manifold of human motion, trained on a large motion capture dataset. The learned motion manifold, which is represented by the hidden units of a convolutional autoencoder, represents motion data in sparse components which can be combined to produce a wide range of complex movements. To map from high level parameters to the motion manifold, we stack a deep feedforward neural network on top of the trained autoencoder. This network is trained to produce realistic motion sequences from parameters such as a curve over the terrain that the character should follow, or a target location for punching and kicking. The feedforward control network and the motion manifold are trained independently, allowing the user to easily switch between feedforward networks according to the desired interface, without re-training the motion manifold. Once motion is generated it can be edited by performing optimization in the space of the motion manifold. This allows for imposing kinematic constraints, or transforming the style of the motion, while ensuring the edited motion remains natural. As a result, the system can produce smooth, high quality motion sequences without any manual pre-processing of the training data.
How to read this
- Category
- Method: a deep-learning framework for motion synthesis and editing
- Contributions
- Learns a motion manifold via a convolutional autoencoder trained on a large motion-capture dataset, representing motion as sparse combinable components
- Stacks a feedforward network on top of the manifold to map high-level parameters (terrain curves, target locations for punching/kicking) to realistic motion
- Enables editing by optimizing in the manifold space, imposing kinematic constraints or transforming style while keeping motion plausible
- Context
- Extends the authors' earlier convolutional-autoencoder motion manifold work (holden-deep-learning-motion-2015) into a full synthesis-and-editing framework with high-level control.Builds on: Learning Motion Manifolds with Convolutional Autoencoders
- Correctness
- Assumes the learned manifold captures the space of natural human motion from the training corpus, so synthesized and edited motion stays plausible only within that distribution; out-of-distribution parameters or styles may degrade results.
- Clarity
- Reasonably accessible; a first pass conveys the manifold-plus-control idea, a second pass clarifies the autoencoder architecture and the editing optimization.
- How to read it
- Focus first on the separation of motion manifold from control network (trained independently); a second pass is worth it to understand the convolutional autoencoder and the manifold-space optimization for editing.
Built upon by
Related work
- Phase-Functioned Neural Networks for Character Control 2017 / SIGGRAPH
- Learning Motion Manifolds with Convolutional Autoencoders 2015 / SIGGRAPH Asia
- Character Controllers Using Motion VAEs 2020 / SIGGRAPH
- Neural State Machine for Character-Scene Interactions 2019 / SIGGRAPH Asia
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →