← ArchivePaper2015

Learning Motion Manifolds with Convolutional Autoencoders

Daniel Holden, Taku Komura, Jun Saito, Ikhsanul Habibie

SIGGRAPH AsiaAcademic312 cites26 descendantsMotion Synthesis

Convolutional autoencoder learning a compact motion manifold from raw motion capture data for synthesis and style transfer.

Open the paper ↗Trace in the lineage graph

Abstract

This technical brief presents a method for learning a manifold of human motion data using a deep convolutional autoencoder trained on the complete CMU motion capture database. Motion is represented as a time series of joint positions, and the network learns forward and inverse projection operators that map data onto a bounded motion manifold using one-dimensional temporal convolution, max pooling, and denoising autoencoding. The learned manifold acts as a prior over valid human motion and supports applications such as fixing corrupted or noisy Kinect captures, filling in missing marker data, interpolating between motions without blending artefacts, and computing temporally invariant distances between motions. The approach scales to large datasets with little preprocessing and projects motion in a fraction of a millisecond at runtime.

How to read this

Category: Method: learned motion manifold (convolutional autoencoder)
Contributions: Trains a deep convolutional autoencoder on the full CMU mocap database to learn a bounded motion manifold acting as a prior over valid human motion.
Learns forward and inverse projection operators using 1D temporal convolution, max pooling and denoising autoencoding.
Supports cleanup of corrupted/noisy Kinect captures, missing-marker fill-in, artefact-free interpolation, and temporally invariant motion distances at sub-millisecond runtime.
Context: An early deep-learning approach to data-driven motion modelling, representing motion as joint-position time series and using the manifold as a learned motion prior.
Correctness: The manifold is only as broad as the CMU data it is trained on, so out-of-distribution or highly stylized motion may project poorly; presented as a technical brief, so treat the application demos as illustrative rather than exhaustively evaluated.
Clarity: Accessible as a brief; a first pass conveys the manifold-prior idea, a second pass clarifies the temporal-convolution architecture.
How to read it: First pass for the manifold-as-prior concept and its applications; second pass on the autoencoder structure and projection operators if building motion-synthesis tooling.

Builds on

Nothing in the archive, this is a starting point.

Built upon by

Related work

A Deep Learning Framework for Character Motion Synthesis and Editing 2016 / SIGGRAPH
Neural Animation Layering for Synthesizing Martial Arts Movements 2021 / SIGGRAPH
Data-Driven Autocompletion for Keyframe Animation 2018 / MIG
Dynamic Hair Modeling from Monocular Videos Using Deep Neural Networks 2019 / SIGGRAPH Asia

Keywords

convolutional autoencoder motion manifold manifold learning deep neural networks convolutional neural networks denoising autoencoder cmu motion capture database human motion max pooling motion interpolation motion data corruption geodesic distance unsupervised learning temporal convolution theano prior probability distribution

This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →