ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes

How to read this

Category

generative pose space discovery paper for auto rigged meshes

Contributions

• Introduces Video-informed Pose Spaces, a feedforward model that discovers a plausible, navigable pose space manifold for any auto rigged mesh by distilling motion priors from a pretrained video diffusion model (Wan2.2-TurboDiffusion), rather than relying on scarce artist authored 4D data
• Applies differentiable geometric validators to the skinned mesh to enforce shape specific plausibility, such as avoiding hyperextension and self intersection, without hand tuned regularisers
• Enables three concrete applications from the same learned pose space: manifold constrained semantic pose editing, smooth pose space interpolation for keyframing, and using sampled 3D poses as guidance for video generation
• Releases a 4D motion dataset of 127,000 poses spanning over 100 species and 200 individuals, built from generative video priors with VLM guidance and 4D reconstruction using RigAnything and ActionMesh
• Reports matching the plausibility and diversity of state of the art models trained on curated synthetic 4D data (Objaverse-XL, Truebones), plus zero shot generalisation to unseen species and skeletal topologies

Context

ViPS sits between two extremes it names directly: rigging, which gives control but no plausibility bounds, and video diffusion priors, which capture plausible motion but offer no rig level control. It also positions itself against 4D reconstruction methods such as ActionMesh and the concurrent RigMo, which recover individual motion instances rather than a navigable manifold, and it builds directly on RigAnything for auto rigging and on ActionMesh for turning video into 4D motion guidance used as training supervision.

Correctness

The main experiments distil a single specific video model (Wan2.2-TurboDiffusion), with a supplemental check that other video models such as Kling, RunWay and SeedDance produce qualitatively similar pose optimisation results. Because the training data is generated and reconstructed from video rather than captured directly, plausibility is bounded by both the geometric validators and by whatever biases the underlying video model carries, so systematic artifacts in the video prior could propagate into the learned pose space.

Clarity

Dense research paper prose aimed at a generative 3D graphics research audience, but the core idea, using a video prior as a substitute for scarce 4D training data, is explainable to a rigging savvy reader even without a diffusion modelling background.

How to read it

First pass, read the abstract, the Figure 1 teaser, and the three listed applications to see what a discovered pose space buys you in practice. Second pass, read the introduction's framing of rigging versus video priors and the data pipeline figures to see how video is turned into training supervision. Third pass, read the quantitative evaluation section for the pose validity, manifold coverage and temporal smoothness metrics compared against Objaverse-XL and Truebones trained baselines, since that is what actually backs the matches state of the art claim.