← ArchivePaper2021
S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling
Ze Yang, Shenlong Wang, Sivabalan Manivasagam, Zeng Huang, Wei-Chiu Ma, Xinchen Yan, Ersin Yumer, Raquel Urtasun
Jointly learns implicit shape, skeleton topology, and skinning weight fields from data, enabling animation and novel view synthesis from RGB or LiDAR input.
Abstract
Constructing and animating humans is an important component for building virtual worlds in a wide variety of applications such as virtual reality or robotics testing in simulation. As there are exponentially many variations of humans with different shape, pose and clothing, it is critical to develop methods that can automatically reconstruct and animate humans at scale from real world data. Towards this goal, we represent the pedestrian’s shape, pose and skinning weights as neural implicit functions that are directly learned from data. This representation enables us to handle a wide variety of different pedestrian shapes and poses without explicitly fitting a human parametric body model, allowing us to handle a wider range of human geometries and topologies. We demonstrate the effectiveness of our approach on various datasets and show that our reconstructions outperform existing state-of-the-art methods. Furthermore, our re-animation experiments show that we can generate 3D human animations at scale from a single RGB image (and/or an optional LiDAR sweep) as input.
How to read this
- Category
- Method: neural implicit human modeling (shape, skeleton, skinning fields)
- Contributions
- Jointly represents a person's shape, skeleton/pose, and skinning weights as neural implicit functions learned directly from data
- Avoids explicitly fitting a parametric body model, handling a wider range of human geometries and topologies
- Reconstructs and re-animates 3D humans at scale from a single RGB image with an optional LiDAR sweep
- Context
- Positions itself relative to parametric body models like SMPL (Loper et al.), replacing the fixed template with learned implicit shape, skeleton, and skinning fields.Builds on: SMPL: A Skinned Multi-Person Linear Model
- Correctness
- Demonstrated on pedestrian-style datasets with RGB and optional LiDAR input and reported to outperform prior reconstructions; as a learned implicit method, output fidelity depends on the training data distribution and on how much the optional LiDAR sweep contributes, which a reader should weigh.
- Clarity
- Technical but well-motivated; a first pass conveys the three-field decomposition and the template-free argument, a second pass is needed for the field formulations and training.
- How to read it
- First pass for why dropping the parametric template helps with diverse geometries; do a second pass on the skeleton and skinning field definitions if you build implicit animatable humans.
Builds on
Built upon by
Nothing yet.
Related work
- One Model to Rig Them All: Diverse Skeleton Rigging with UniRig 2025 / SIGGRAPH
- Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies 2021 / CVPR
- Learning Skeletal Articulations with Neural Blend Shapes 2021 / SIGGRAPH
- MoRig: Motion-Aware Rigging of Character Meshes from Point Clouds 2022 / SIGGRAPH Asia
Keywords
This page summarises the entry and links to its original source. The archive never hosts or redistributes the publication itself.Show it in the full archive list →