Papers.

CANRig is a fully automated neural facial rigging method that lets artists deform a 3D face mesh by manipulating sparse control handles with continuous, user-defined locality at runtime. It formulates deformation as a cross-attention between control handles and mesh vertices, where the attention matrix is masked by per-vertex influence weights derived from a user-specified radius and falloff, enabling smooth transitions from precise local edits to broad global changes. A shape-preserving workflow built from zero-preserving MLPs, an added base-shape attention column, and a shape-preserving loss guarantees that previous edits are maintained during iterative, non-destructive editing and yields zero-error inversion of captured performances. For high-resolution live-action meshes the method predicts local patch blendshape weights instead of per-vertex deltas, and the authors demonstrate posing, clip editing, dialogue replacement, and expression transfer across both feature animation and visual effects pipelines.

Related Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild · Example-Based Facial Rigging · Reusable Facial Rigging and Animation: Create Once, Use Many · Dynamic 3D Avatar Creation from Hand-Held Video Input

Despite transformative advances in generative motion synthesis, real-time interactive motion control remains dominated by traditional techniques. In this work, we identify two key challenges in bridging research and production: 1) Real-time scalability: Industry applications demand real-time generation of a vast repertoire of motion skills, while generative methods exhibit significant degradation in quality and scalability under real-time computation constraints, and 2) Integration: Industry applications demand fine-grained multi-modal control involving velocity commands, style selection, and precise keyframes, a need largely unmet by existing text- or tag-driven models. Moreover, a systematic motion design interface for generative models remains absent. To overcome these limitations, we introduce MotionBricks: a large-scale, real-time generative framework with a two-fold solution. First, we propose a large-scale modular latent generative backbone tailored for robust real-time motion generation, effectively modeling a dataset of over 350,000 motion clips with a single model. Second, we introduce smart primitives that provide a unified, robust, and intuitive interface for authoring both navigation and object interaction. Notably, MotionBricks applies to new downstream tasks in a zero-shot manner, where no finetuning or task-specific tagging is required. Applications can be designed in a plug-and-play manner like assembling bricks without expert animation knowledge, enabling an accessible interface for applications in animation and robotics. Quantitatively, we show that MotionBricks produces state-of-the-art motion quality on open-source and proprietary datasets of various scales, while also achieving a real-time throughput of 15,000 FPS with 2ms latency. We demonstrate the flexibility and robustness of MotionBricks in a complete production-level animation demo, covering navigation and object-scene interaction across various styles with a unified model. To showcase our framework's application beyond animation, we deploy MotionBricks on the Unitree G1 humanoid robot to demonstrate its flexibility and generalization for real-time robotic control.

Related Character Motion Synthesis by Topology Coordinates · Mode-Adaptive Neural Networks for Quadruped Motion Control · Motion Retargeting for Crowd Simulation · SMPL: A Skinned Multi-Person Linear Model

Parametric human body models are foundational to human reconstruction, animation, and simulation, yet they remain mutually incompatible: SMPL, SMPL-X, MHR, Anny, and related models each diverge in mesh topology, skeletal structure, shape parameterization, and unit convention, making it impractical to exploit their complementary strengths within a single pipeline. We present SOMA, a unified body layer that bridges these heterogeneous representations through three abstraction layers. Mesh topology abstraction maps any source model's identity to a shared canonical mesh in constant time per vertex. Skeletal abstraction recovers a full set of identity-adapted joint transforms from any body shape, whether in rest pose or an arbitrary posed configuration, in a single closed-form pass, with no iterative optimization or per-model training. Pose abstraction inverts the skinning pipeline to recover unified skeleton rotations directly from posed vertices of any supported model, enabling heterogeneous motion datasets to be consumed without custom retargeting. Together, these layers reduce the $O(M^2)$ per-pair adapter problem to $O(M)$ single-backend connectors, letting practitioners freely mix identity sources and pose data at inference time. The entire pipeline is fully differentiable end-to-end and GPU-accelerated via NVIDIA-Warp.

In this Autodesk University 2025 live demo, CEO Andrew Anagnost and product manager Lance Thornton show MotionMaker fully integrated into Maya generating plausible human locomotion from just a couple of spatial keyframes via live neural-network inference, trained on a large set of human motion data so it reproduces weight shifts, arm swing, and walk-to-run transitions. They demonstrate art-directing the result by adding a keyframe to redirect the character around a corner and regenerating on the fly. The session also previews FaceAnimator, which learns a character's facial and lip motion from existing animated sequences of a show and then generates face animation directly from an input audio file, and teases Autodesk Assistant scene editing in Maya driven by MCP servers.

Related Meet MotionMaker: New AI Animation Tool In Maya · Audio-Driven Facial Animation by Joint End-to-End Learning of Pose and Emotion · Dog Code: Human to Quadruped Embodiment Using Shared Codebooks · Mode-Adaptive Neural Networks for Quadruped Motion Control

Despite the growing accessibility of skeletal motion data, integrating it for animating character meshes remains challenging due to diverse configurations of both skeletons and meshes. Specifically, the body scale and bone lengths of the skeleton should be adjusted in accordance with the size and proportions of the mesh, ensuring that all joints are accurately positioned within the character mesh. Furthermore, defining skinning weights is complicated by variations in skeletal configurations, such as the number of joints and their hierarchy, as well as differences in mesh configurations, including their connectivity and shapes. While existing approaches have made efforts to automate this process, they hardly address the variations in both skeletal and mesh configurations. In this paper, we present a novel method for the automatic rigging and skinning of character meshes using skeletal motion data, accommodating arbitrary configurations of both meshes and skeletons. The proposed method predicts the optimal skeleton aligned with the size and proportion of the mesh as well as defines skinning weights for various meshskeleton configurations, without requiring explicit supervision tailored to each of them. By incorporating Diffusion 3D Features (Diff3F) as semantic descriptors of character meshes, our method achieves robust generalization across different configurations.

Related One Model to Rig Them All: Diverse Skeleton Rigging with UniRig · SkinMixer: Blending 3D Animated Models · Mobilizing Mocap, Motion Blending, and Mayhem: Rig Interoperability for Crowd Simulation on Incredibles 2 · Automatic Rigging and Animation of 3D Characters

Parametric body models offer expressive 3D representation of humans across a wide range of poses, shapes, and facial expressions, typically derived by learning a basis over registered 3D meshes. However, existing human mesh modeling approaches struggle to capture detailed variations across diverse body poses and shapes, largely due to limited training data diversity and restrictive modeling assumptions. Moreover, the common paradigm first optimizes the external body surface using a linear basis, then regresses internal skeletal joints from surface vertices. This approach introduces problematic dependencies between internal skeleton and outer soft tissue, limiting direct control over body height and bone lengths. To address these issues, we present ATLAS, a high-fidelity body model learned from $600 k$ high-resolution scans captured using 240 synchronized cameras. Unlike previous methods, we explicitly decouple the shape and skeleton bases by grounding our mesh representation in the human skeleton. This decoupling enables enhanced shape expressivity, fine-grained customization of body attributes, and keypoint fitting independent of external soft-tissue characteristics.

Related SUPR: A Sparse Unified Part-Based Human Representation · Expressive Body Capture: 3D Hands, Face, and Body from a Single Image · STAR: Sparse Trained Articulated Human Body Regressor · SoftSMPL: Data-driven Modeling of Nonlinear Soft-tissue Dynamics for Parametric Humans

Framestore breaks down its character and effects work on Netflix's live-action Avatar: The Last Airbender, focusing on grounding the sky bison Appa with a realistic fur groom and designing the creature Momo (a blend of flying lemur, bat and monkey referenced against fennec foxes and monkeys). The team describes using Houdini across the show for Appa's fur, digi-doubles, cloth dynamics and muscle simulation, plus a large storm sequence requiring water, rain and cloud volumetric simulations. They emphasize Houdini's node-based flexibility for blending fluid and particle simulations together with fur simulation for cleaner integration. Animation exploration tests such as walk cycles and grabbing actions were used to balance Momo's oversized cartoon ears against believable motion.

Related Simulating the Perfect Groom for a Bovine Biker | Untold Studios | FMX HIVE 2023 · Creating a Photorealistic Hyena · Creatures in Houdini | Ahmed Gharraph | FMX 2019 · Framestore Creatures & Houdini | Framestore | Character FX & Crowds Production Talks

Matthew Tucker introduces Autodesk's Bifrost rigging module system, a fully procedural compound-based framework for building modular character rigs inside Maya. Each rig part is a self-contained module that encapsulates joint and control hierarchies plus transformation logic, connected through standardized input, output, and parent ports. Pins act as guide transforms that define the rest pose and drive the pivot matrix, and because the system is procedural, pins can be moved after animation without breaking the rig or requiring a rebuilt bind pose for skin clusters. Transforms share a core type with automatic propagation, animation is driven via operator matrices, and custom behavior such as inverse kinematics is authored in the user animation compound, with the included arm module offered as a reference example.

Related Bifrost Rigging in Maya - Part 2: The Module Interface · Bifrost Rigging in Maya - Part 3: Creating a Custom Module · Deliver Faster Rigging and Animation with AI · Group Based Rigging of Realistically Feathered Wings

Matthew Tucker walks through the Bifrost rigging module interface in Maya, creating a new graph in the Bifrost Graph Editor and explaining its left-to-right data flow through inputs, outputs, and parent ports that link parent and child modules. Using the parameter editor he covers the module name and state parameters, showing how an FK chain behaves under the animation, rest relative, and rest absolute states and how a flattened versus direct hierarchy affects inherited transformations. He then demonstrates diagnostics mode for visualizing pin, control, and joint orientations via display transforms, and uses profile evaluation with the Maya profiler to measure how optimized a variable FK module is.

Related Bifrost Rigging in Maya - Part 1: Introduction and Framework · Bifrost Rigging in Maya - Part 3: Creating a Custom Module · Digital Humans: Inside Epic's MetaHuman Creator · Deliver Faster Rigging and Animation with AI

Matthew Tucker walks through building a custom spline rig module inside Maya's Bifrost Modular Rigging Framework, starting from the editable template module and parenting it to a root module. He demonstrates authoring base and end controls with pins, then using iterate nodes to procedurally generate any number of evenly spaced mid controls and joints, driving their placement with interpolate transform matrix, divide/increment/decrement nodes, and dynamically built name strings. He builds a spline through the controls with a resample matrix chain, editing its decomposed-matrix internals to manually set the forward axis and up vector so the spline orientation matches the controls, then drives the joint chain along it. The module is cleaned up, parameters like joint count and mid-control count are promoted to the top of the graph, and the optimized module is published for reuse.

Related Bifrost Rigging in Maya - Part 2: The Module Interface · Bifrost Rigging in Maya - Part 1: Introduction and Framework · Digital Humans: Inside Epic's MetaHuman Creator · Deliver Faster Rigging and Animation with AI

Recent advancements in 3D human pose estimation from single-camera images and videos have relied on parametric models, like SMPL. However, these models over-simplify anatomical structures, limiting their accuracy in capturing true joint locations and movements, which reduces their applicability in biomechanics, healthcare, and robotics. Biomechanically accurate pose estimation, on the other hand, typically requires costly marker-based motion capture systems and optimization techniques in specialized labs. To bridge this gap, we propose BioPose, a novel learning-based framework for predicting biomechanically accurate 3D human pose directly from monocular videos. BioPose includes three key components: a Multi-Query Human Mesh Recovery model (MQ-HMR), a Neural Inverse Kinematics (NeurIK) model, and a 2D-informed pose refinement technique. MQ-HMR leverages a multi-query deformable transformer to extract multi-scale fine-grained image features, enabling precise human mesh recovery. NeurIK treats the mesh vertices as virtual markers, applying a spatial-temporal network to regress biomechanically accurate 3D poses under anatomical constraints. To further improve 3D pose estimations, a 2D-informed refinement step optimizes the query tokens during inference by aligning the 3D structure with 2D pose observations.

Related SFV: Reinforcement Learning of Physical Skills from Video · AddBiomechanics: Automating model scaling, inverse kinematics, and inverse dynamics from human motion data through sequential optimization · From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans · Reconstructing Humans with a Biomechanically Accurate Skeleton

BlendSim shifts physics-based animation from discrete frame-by-frame solves to continuous interpolation trajectories built on blendshapes, representing each vertex path with continuous parametric Bezier splines that have variable keyframe times. Because this mesh animation representation is continuous and fully differentiable, it can be optimized to follow the laws of physics under constraints, while projective dynamics decouples the optimization into local parallelizable steps and a global quadratic step for efficiency. The method supports constraints such as collisions and cyclic motion and is compatible with modern animation workflows and file formats such as glTF.

Related An Implicit Physical Face Model Driven by Expression and Style · Gravity Preloading for Maintaining Hair Shape Using the Simulator as a Closed-Box Function · Clean Cloth Inputs: Removing Character Self-Intersections with Volume Simulation · Hyper-Reduced Projective Dynamics

In Walt Disney Animation Studios’ "Moana 2", the main characters undergo significant evolution from the original film, displaying a broad range of emotions while interacting with diverse environmental conditions. Crafting the hair and cloth motion to consistently support and enhance the complex character performances presented challenges for the character team, particularly when combined with nuanced art direction. To this end, we developed a multi-tiered and strategic approach for the choreography of the hair and cloth which includes three main areas of emphasis: Performance Categorization, Continuity through Visual Planning, and Iterative Refinement. Integral to this process were extensive and detailed drawovers, to ensure that hairstyles and costumes reacted consistently. This strategy allowed us to frontload the work through careful planning, enabling technical animation to execute on art direction of shots efficiently. We illustrate the overall approach with specifics for hair with Moana and Maui, and with Matangi for cloth.

Related Scriptable Character FX Solution · Simulating Wind Effects on Cloth and Hair in Disney's Frozen · Art-Directing Asha's Braids in Disney's Wish · Anisotropic Elastoplasticity for Cloth, Knit and Hair Frictional Contact

Katana’s [Foundry 2025a] strength is its procedural node graph architecture which provides dynamic and scalable workflows - studios are no strangers to reaching scenes containing tens of thousands of nodes. The layer-based nature of Universal Scene Description (USD) [Pixar Anmation Studios 2025] presents a challenge for the node-based procedural approach. At large scales, the current procedural method of creating a USD layer per node is inadvisable because modifying an early node could trigger a full USD Stage recomposition and reprocessing of USD layer generating functions. In this extended abstract, an insight into the journey towards our solution is presented: the UsdSuperLayer node type. Providing a “procedural on-demand” approach, enabling efficient management of thousands of primitives and extensive shading graphs represented as nodes.

Related Universal Scene Description: Open Source Release · A Deep Dive into Universal Scene Description and Hydra · Forging a New Animation Pipeline with USD · Using USD in Pixar's Digital Backlot

Cloth draping is a prevalent tailoring process that gives 3D form to sewn 2D panels of fabric. However, when dressing animated characters, artists often prefer to model garments with delineated spatial structures and clean silhouettes at the cost of diminishing the presence of folds and wrinkles produced by draping. To reconcile stylization and realism, this work describes a new approach for directing cloth draping that accommodates 3D shaping and 2D pattern making simultaneously. Our key contribution is a method that generates custom UVs blending the distortion induced by 3D shapes into 2D fabric panels. As a result, we can retarget cloth simulations to compute physically plausible draping deformations that smoothly transition to prescribed 3D forms. To assist the garment design, we also propose a flattening tool that constructs low-resolution UV panels amenable to 2D manipulation. We showcase our results with a series of garment assets and cloth animations from Pixar feature films Inside Out 2 (2024) and Elio (2025).

Related Art-Directed Costumes at Pixar: Design, Tailoring, and Simulation in Production · Mixing Yarns and Triangles in Cloth Simulation · Untangling Cloth · Multi-Resolution Isotropic Strain Limiting

This quick-start tutorial walks through MotionMaker in Maya 2026.1, showing how to generate character locomotion from a drawn path instead of extensive keyframing or mocap. The presenter imports a default biped, keys a path locator across the scene at various frames, and clicks Generate Motion to calculate and cache the animation. It then demonstrates editing the path to route around obstacles using the AutoSpeedRamp function, walking up a hill by raising final frames, and adding a jump via right-click action tags on the movement bar. The final section retargets the generated motion onto a HumanIK MayaBot character through the Character Control tab using the MoMo Standard Biped as the source.

Related Meet MotionMaker: New AI Animation Tool In Maya · MotionBuilder: Essentials Characterization, Retargeting and Baking Animations · Motion Builder Characterization and Retargeting Tutorial · Physically Based Motion Transformation

Parametric body models provide the structural basis for many human-centric tasks, yet existing models often rely on costly 3D scans and learned shape spaces that are proprietary and demographically narrow. We introduce Anny, a simple, fully differentiable, and scan-free human body model grounded in anthropometric knowledge from the MakeHuman community. Anny defines a continuous, interpretable shape space, where phenotype parameters (e.g. gender, age, height, weight) control blendshapes spanning a wide range of human forms--across ages (from infants to elders), body types, and proportions. Calibrated using WHO population statistics, it provides realistic and demographically grounded human shape variation within a single unified model. Thanks to its openness and semantic control, Anny serves as a versatile foundation for 3D human modeling--supporting millimeter-accurate scan fitting, controlled synthetic data generation, and Human Mesh Recovery (HMR). We further introduce Anny-One, a collection of 800k photorealistic images generated with Anny, showing that despite its simplicity, HMR models trained with Anny can match the performance of those trained with scan-based body models. The Anny body model and its code are released under the Apache 2.0 license, making Anny an accessible foundation for human-centric 3D modeling.

With the rapid evolution of 3D generation algorithms, the cost of producing 3D humanoid character models has plummeted, yet the field is impeded by the lack of a comprehensive dataset for automatic rigging, a pivotal step in character animation. Addressing this gap, we present HumanRig, the first large-scale dataset specifically designed for 3D humanoid character rigging, encompassing 11,434 meticulously curated T-posed meshes adhered to a uniform skeleton topology. Capitalizing on this dataset, we introduce an innovative, data-driven automatic rigging framework, which overcomes the limitations of GNNbased methods in handling complex AI-generated meshes. Our approach integrates a Prior-Guided Skeleton Estimator (PGSE) module, which uses 2D skeleton joints to provide a preliminary 3D skeleton, and a Mesh-Skeleton Mutual Attention Network (MSMAN) that fuses skeleton features with 3D mesh features extracted by a U-shaped point transformer. This enables a coarse-to-fine 3D skeleton joint regression and a robust skinning estimation, surpassing previous methods in quality and versatility. This work not only remedies the dataset deficiency in rigging research but also propels the animation industry towards more efficient and automated character rigging pipelines.

Related RigNet: Neural Rigging for Articulated Characters · Automatic Rigging and Animation of 3D Characters · Pose and Skeleton-aware Neural IK for Pose and Motion Editing · MoRig: Motion-Aware Rigging of Character Meshes from Point Clouds

Augments a facial animation rig with real-time shell simulation and contact handling, without anatomical priors or additional artist intervention. Layered over an existing rig, it adds physically plausible skin dynamics and self-contact at the lips and eyelids, runs interactively on consumer hardware, and generalises across realistic, stylized and fantastical characters.

Related High Fidelity Facial Animation Capture and Retargeting with Contours · Position Based Dynamics · Enriching Facial Blendshape Rigs with Physical Simulation · Fast Contact Determination for Intersecting Deformable Solids

Framestore's head of groom and creature FX gives an end-to-end tour of building digital animals in Houdini, framing CFX as the last department before render and distinguishing it from FX and from grooming. She emphasizes preparation and reusable HDA templates that handle scaling, colliders, pre-roll, time remapping and farm publishing, then walks through the standard simulation stack: muscle tension and impact computed via stretching lines and a custom SOP solver, tetrahedron attachment with delta-mesh correction, and solving in Vellum, FEM or third-party solvers. She covers three fat approaches (target-based, attachment-based, and fascia-and-collision), tetrahedron creation with solidembed versus tetconform plus FEM validate for inverted tets, frame-by-frame recreated constraints for skin sliding, FEM-driven wrinkles, and cloth draping with Marvelous Designer. The hair section uses guide-deform to glue guides to characters and back to the groom, custom two-channel wind forces, and FLIP-based water interaction, finishing with procedural shot sculpting and career advice on VEX and Python.

Related Stags & Stripes, Creating Photoreal Characters | Framestore | FMX HIVE Europe 2021 · The Making of CG Animals · Simulating the Perfect Groom for a Bovine Biker | Untold Studios | FMX HIVE 2023 · Creating a Photorealistic Hyena

This talk introduces MotionMaker, a generative locomotion tool built into Maya and powered by Autodesk AI that combines machine learning, motion capture and keyframing to synthesize believable biped and quadruped motion. Animators create a character, set keyframes in space or draw a motion path, then hit generate to produce base motion that can be refined in MotionMaker's dedicated editor with action tags such as jump or sit. The system retargets standard biped or quadruped motion onto custom characters using HIK or custom retargeting solutions, and layers further edits like head turns via animation layers. It also covers path modes for locking motion to timing, speed ramping to move characters faster between points, and motion scale to convey weight for large characters.

Related Generate animation with AI using Motion Maker in Maya 2026.1 · Autodesk MotionBuilder 2022 · MotionBuilder: Essentials Characterization, Retargeting and Baking Animations · Motion Builder Characterization and Retargeting Tutorial

We present a novel character rigging solution developed for OOOOO, a liquid supercomputer in Pixar’s Elio. OOOOO’s design and desired movement necessitated reimagining our conventional way of articulating characters and she became Pixar’s first mesh-free character rig. We developed a system that allowed our animators full fidelity control over what is essentially a rigged shader while ensuring downstream renderability. [Luo et al. 2025] The system’s architecture supports a hierarchical arrangement of implicit surface primitives and operators, allowing for complex transformations while preserving normal animation paradigms and offers unprecedented flexibility in character animation.

Related Eyes Without a Face: Integrating Detached Facial Features into Pixar's Character Pipeline · Shaping the Elements: Curvenet Animation Controls in Pixar's Elemental · Making Souls: Methods and a Pipeline for Volumetric Characters · Mobilizing Mocap, Motion Blending, and Mayhem: Rig Interoperability for Crowd Simulation on Incredibles 2

We present MHR, a parametric human body model that combines the decoupled skeleton/shape paradigm of ATLAS with a flexible, modern rig and pose corrective system inspired by the Momentum library. Our model enables expressive, anatomically plausible human animation, supporting non-linear pose correctives, and is designed for robust integration in AR/VR and graphics pipelines.

This work addresses generating facial blendshapes and reference animations for a new 3D character in production settings where expressions and animations already exist on a predefined template character. The authors propose Neural Facial Deformation Transfer (NFDT), a data-driven method that transfers facial expressions from a template character to new target characters given only the target's neutral shape. They introduce a data generation strategy that automatically builds a large training dataset of paired template and target shapes in the same expression, then train a topology-agnostic decoder-only transformer adapted from the Shape Transformer to perform the transfer in high fidelity. NFDT operates without a rig inversion step, generalizes across varying mesh topologies and to humanoid creatures, and outperforms prior facial expression transfer methods in quantitative evaluations and a user study.

Related Transferring Facial Expressions to Different Face Models · Facial Retargeting with Automatic Range of Motion Alignment · Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild · Local Anatomically-Constrained Facial Performance Retargeting

The rapid evolution of 3D content creation, encompassing both AI-powered methods and traditional workflows, is driving an unprecedented demand for automated rigging solutions that can keep pace with the increasing complexity and diversity of 3D models. We introduce UniRig, a novel, unified framework for automatic skeletal rigging that leverages the power of large autoregressive models and a bone-point cross-attention mechanism to generate both high-quality skeletons and skinning weights. Unlike previous methods that struggle with complex or non-standard topologies, UniRig accurately predicts topologically valid skeleton structures thanks to a new Skeleton Tree Tokenization method that efficiently encodes hierarchical relationships within the skeleton. To train and evaluate UniRig, we present Rig-XL, a new large-scale dataset of over 14,000 rigged 3D models spanning a wide range of categories. UniRig significantly outperforms state-of-the-art academic and commercial methods, achieving a 215% improvement in rigging accuracy and a 194% improvement in motion accuracy on challenging datasets. Our method works seamlessly across diverse object categories, from detailed anime characters to complex organic and inorganic structures, demonstrating its versatility and robustness.

Related S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling · ASMR: Adaptive Skeleton-Mesh Rigging and Skinning via 2D Generative Prior · MoRig: Motion-Aware Rigging of Character Meshes from Point Clouds · NeuroSkinning: Automatic Skin Binding for Production Characters with Deep Graph Networks

We introduce Pseudo-Collisions (PC), a numerical and time independent method to reduce the collisions between short fur and the skin of an asset as it deforms during animation. Typically, solving these intersections in a standard workflow would require a lot of time to set up and simulate each strand or their guide curves. With our solution, this can be achieved with a light post-process on top of the applied fur. This is achieved by adjusting the transformation matrix used to place the strands on the animated skin so that its rotation component smoothly lifts the hair where necessary. The algorithm dynamically adjusts the orientation of the animated strands to reduce the intersections using different pieces of information including the direction of the strands, the placement matrices for each strand on the rest and the animated skin meshes, as well as the change in the surface’s curvature due to the deformation. Pseudo-Collisions was successfully used during the production of Mufasa: The Lion King [Disney© 2024] and the set of parameters made available to the users to modulate the results of the correction is also presented.

Related Creatures in Houdini | Ahmed Gharraph | FMX 2019 · Framestore Creatures & Houdini | Framestore | Character FX & Crowds Production Talks · Hair and Fur in an Evolving Pipeline · Cloth and Skin Deformation with a Triangle Mesh Based Convolutional Neural Network

HSMR is the first end-to-end approach for reconstructing 3D humans from a single image by estimating the parameters of the biomechanically accurate SKEL model. A transformer is trained to estimate SKEL parameters from image inputs. Due to the lack of training data, the authors build a pipeline to produce pseudo ground truth model parameters and implement a training procedure that iteratively refines them. HSMR achieves competitive performance on standard benchmarks and significantly outperforms prior methods in settings with extreme 3D poses and viewpoints, while producing more realistic joint rotation estimates by respecting anatomical joint limits. Accepted as CVPR 2025 Oral.

Related BioPose: Biomechanically-accurate 3D Pose Estimation from Monocular Videos · SMPL: A Skinned Multi-Person Linear Model · BOSS: Bones, Organs and Skin Shape Model · 3D Mesh Pose Transfer Based on Skeletal Deformation

This paper introduces a rest shape optimization framework that produces sag-free simulations of discrete elastic rods under gravity. Rather than adjusting stiffness or applying compensating forces, the method optimizes per-element rest shape parameters so that the input configuration is an equilibrium state of the simulated rod. The resulting formulation is compatible with existing discrete elastic rod solvers and avoids visually undesirable initial sagging for hair, cables, and similar thin elastic structures.

Related Gravity Preloading for Maintaining Hair Shape Using the Simulator as a Closed-Box Function · Simulation-Ready Hair Capture · Efficient and Stable Approach to Elasticity and Collisions for Hair Animation · Discrete Elastic Rods

In this paper, we present RigAnyFace (RAF), a scalable neural auto-rigging framework for facial meshes of diverse topologies, including those with multiple disconnected components. RAF deforms a static neutral facial mesh into industry-standard FACS poses to form an expressive blendshape rig. Deformations are predicted by a triangulation-agnostic surface learning network augmented with our tailored architecture design to condition on FACS parameters and efficiently process disconnected components. For training, we curated a dataset of facial meshes, with a subset meticulously rigged by professional artists to serve as accurate 3D ground truth for deformation supervision. Due to the high cost of manual rigging, this subset is limited in size, constraining the generalization ability of models trained exclusively on it. To address this, we design a 2D supervision strategy for unlabeled neutral meshes without rigs. This strategy increases data diversity and allows for scaled training, thereby enhancing the generalization ability of models trained on this augmented data. Extensive experiments demonstrate that RAF is able to rig meshes of diverse topologies on not only our artist-crafted assets but also in-the-wild samples, outperforming previous works in accuracy and generalizability.

Related Creating an Actor-Specific Facial Rig from Performance Capture · Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild · Dynamic 3D Avatar Creation from Hand-Held Video Input · Dynamic Facial Asset and Rig Generation from a Single Scan

The DreamWorks Curvy-Limb rig allows animators to pose complex curves without sacrificing intuitive usability. The system adds additional flexibility to character limbs (arms and legs) and digits (fingers and toes) while retaining the standard arm, leg, and digit controls to which animators are already familiar. Animators can activate the extra controls as needed on the fly without having to blend into any special "states." The robustness of the Curvy-Limb has proven to be instrumental to the character performances of multiple DreamWorks Animation productions, including Ruby Gillman, Teenage Kraken, Trolls Band Together, and The Wild Robot.

Related Shaping the Elements: Curvenet Animation Controls in Pixar's Elemental · How the Rig Design Impacts the Animation Process · Premo: Powerful Character Rigging, Fast Animation · LibEE: A Multithreaded Dependency Graph for Character Animation

We present Rumba Rig, a new rigging framework that enables riggers to work directly on the final rig graph in a modular and non-linear way, without the need of auto-rig script or abstraction layer. By reducing complexity, the framework makes rigging more accessible and easier to maintain. We demonstrate through practical examples that Rumba Rig is able to express complex rig constructions and delivers professional-quality rigs.

Related Maya 2020 | Proximity Wrap Deformer · Learning an Inverse Rig Mapping for Character Animation · Stable and Efficient Differential IK · FaceLab: Scalable Facial Performance Capture for Visual Effects

We present a suite of techniques from our in-house simulation framework, Loki, addressing the pervasive challenge of collision instabilities in character effects, particularly in cases where nonphysical pinching prevents collision resolution. We introduce a proximity-tolerant mode for contact projection that trades collision residual for stability, a compliant kinematic mechanism for on-demand gap expansion, and contact-aware strain limiting to prevent penetrations while enforcing target edge lengths. Additionally, we showcase our tools for collider management, including hierarchical collision exclusion, one-sided collision handling, and paintable collision thickness maps. These techniques collectively demonstrate a robust and intuitive workflow for combining physically based collisions with challenging production animations.

Related Implicit Multibody Penalty-based Distributed Contact · Clean Cloth Inputs: Removing Character Self-Intersections with Volume Simulation · Fast Corotated FEM using Operator Splitting · Hair and Fur in an Evolving Pipeline

Aman Akram (CTO) and Andrea Lacedelli (CFX Supervisor) of Untold Studios present the Houdini CFX workflows for Disney's short The Boy and the Octopus, covering tentacle, skin, and hair simulation under a tight six-month, non-destructive procedural pipeline. The octopus is built as a non-hollow tetrahedral mesh simulated with tet stretch constraints, driven from the inside by a stripped-down tissue-inspired core using metaballs and magnets to push volume, with procedural groups exposed as artist sliders for stiffness and a sucker-attraction system gluing tentacles to the boy's head. They detail a thin double-sided clear-coat wetness layer created via Zbrush displacement, ray-casting and VDB operations and stabilized with a pinned cloth simulation around the eye socket, plus five long-hair grooms simulated with Vellum hair constraints, glues, and touch constraints, and a Mickey hat solved as a cloth sim with stitch and shape-match constraints. Rendering is done in Arnold with grooms, feathers, and hair deformed at render time in C++ to keep the data footprint low and time-to-first-pixel fast on their cloud-based pipeline.

Related Simulating the Perfect Groom for a Bovine Biker | Untold Studios | FMX HIVE 2023 · Creating a Photorealistic Hyena · Creatures in Houdini | Ahmed Gharraph | FMX 2019 · Feathers: From Model to Groom to Render | nineteentwenty | Character FX & Crowds Production Talks

We introduce 3D Gaussian blendshapes for modeling photorealistic head avatars. Taking a monocular video as input, we learn a base head model of neutral expression, along with a group of expression blendshapes, each of which corresponds to a basis expression in classical parametric face models. Both the neutral model and expression blendshapes are represented as 3D Gaussians, which contain a few properties to depict the avatar appearance. The avatar model of an arbitrary expression can be effectively generated by combining the neutral model and expression blendshapes through linear blending of Gaussians with the expression coefficients. High-fidelity head avatar animations can be synthesized in real time using Gaussian splatting. Compared to state-of-the-art methods, our Gaussian blendshape representation better captures high-frequency details exhibited in input video, and achieves superior rendering performance.

Related I M Avatar: Implicit Morphable Head Avatars from Videos · Reconstruction of Personalized 3D Face Rigs from Monocular Video · 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting · Neural Head Avatars from Monocular RGB Videos

We introduce an approach that creates animatable hu-man avatars from monocular videos using 3D Gaussian Splatting (3DGS). Existing methods based on neural radi-ance fields (NeRFs) achieve high-quality novel-viewlnovel-pose image synthesis but often require days of training, and are extremely slow at inference time. Recently, the com-munity has explored fast grid structures for efficient training of clothed avatars. Albeit being extremely fast at training, these methods can barely achieve an interactive ren-de ring frame rate with around 15 FPS. In this paper, we use 3D Gaussian Splatting and learn a non-rigid deformation network to reconstruct animatable clothed human avatars that can be trained within 30 minutes and rendered at real-time frame rates (50+ FPS). Given the explicit nature of our representation, we further introduce as-isometric-as-possible regularizations on both the Gaussian mean vectors and the covariance matrices, enhancing the generalization of our model on highly articulated unseen poses. Experi-mental results show that our method achieves comparable and even better performance compared to state-of-the-art approaches on animatable avatar creation from a monoc-ular input, while being 400x and 250x faster in training and inference, respectively. Please see our project page at https://neuralbodies.github.ioI3DGS-Avatar.

Related Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies · PointAvatar: Deformable Point-Based Head Avatars from Videos · Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans · SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes

3D facial motion retargeting has the advantage of capturing and recreating the nuances of human facial motions and speeding up the time‐consuming 3D facial animation process. However, the facial motion retargeting pipeline is limited in reflecting the facial motion's semantic information (i.e., meaning and intensity), especially when applied to nonhuman characters. The retargeting quality heavily relies on the target face rig, which requires time‐consuming preparation such as 3D scanning of human faces and modeling of blendshapes. In this paper, we propose a facial motion retargeting pipeline aiming to provide fast and semantically accurate retargeting results for diverse characters. The new framework comprises a target face parameterization module based on face anatomy and a compatible source motion interpretation module. From the quantitative and qualitative evaluations, we found that the proposed retargeting pipeline can naturally recreate the expressions performed by a motion capture subject in equivalent meanings and intensities, such semantic accuracy extends to the faces of nonhuman characters without labor‐demanding preparations.

Related Facial Retargeting with Automatic Range of Motion Alignment · Semi-Supervised Video-Driven Facial Animation Transfer for Production · Local Anatomically-Constrained Facial Performance Retargeting · Performance-Driven Facial Animation

Grooming and moving feathers at DreamWorks had been a technically complicated and specialized process, and the feathered characters of Kung Fu Panda 4 and The Wild Robot prompted a re-imagining of the toolset to be more approachable. The modernized system adds a card layout toolset, a new curves procedural, real-time viewport visualization, shot-specific feather controls, crowd LOD features and an automated pipeline, used on hundreds of shots across both films.

Related Hummingbird: DreamWorks Feather System · Mesh-Driven Generation and Animation of Groomed Feathers · Apteryx: Procedural Generation, Sculpting and Grooming of Feathers · Rendertime Procedural Feathers Through Blended Guide Meshes

We present a comprehensive neural network to model the deformation of human soft tissues including muscle, tendon, fat and skin. Our approach provides kinematic and active correctives to linear blend skinning [Magnenat-Thalmann et al. 1989] that enhance the realism of soft tissue deformation at modest computational cost. Our network accounts for deformations induced by changes in the underlying skeletal joint state as well as the active contractile state of relevant muscles. Training is done to approximate quasistatic equilibria produced from physics-based simulation of hyperelastic soft tissues in close contact. We use a layered approach to equilibrium data generation where deformation of muscle is computed first, followed by an inner skin/fascia layer, and lastly a fat layer between the fascia and outer skin. We show that a simple network model which decouples the dependence on skeletal kinematics and muscle activation state can produce compelling behaviors with modest training data burden. Active contraction of muscles is estimated using inverse dynamics where muscle moment arms are accurately predicted using the neural network to model kinematic musculotendon geometry.

Related NeuroSkinning: Automatic Skin Binding for Production Characters with Deep Graph Networks · How to Build a Human: Practical Physics-Based Character Animation · Data-Driven Physics for Human Soft Tissue Animation · NiLBS: Neural Inverse Linear Blend Skinning

This work proposes a practical surface-based appearance model for pennaceous feathers, representing far-field appearance with a BSDF that implicitly captures light scattering from the main biological structures of the shaft, barbs and barbules. It introduces a flexible BSDF for non-iridescent feathers, an elliptical BCSDF accounting for elliptical fiber cross-sections and a non-centered scattering medulla, and an analytic masking term. The model can be applied to any surface and avoids explicit geometric modeling of the internal microstructures required by prior approaches.

Related Rendering Iridescent Rock Dove Neck Feathers · Microstructure-based Appearance Rendering for Feathers · Modeling and Rendering of Realistic Feathers · Biological Modeling of Feathers by Morphogenesis Simulation

Coordinate based implicit neural representations have gained rapid popularity in recent years as they have been successfully used in image, geometry and scene modeling tasks. In this work, we present a novel use case for such implicit representations in the context of learning anatomi-cally constrained face models. Actor specific anatomically constrainedface models are the state of the art in bothfacial performance capture and performance retargeting. Despite their practical success, these anatomical models are slow to evaluate and often require extensive data capture to be built. We propose the anatomical implicit face model; an ensem-ble of implicit neural networks that jointly learn to model the facial anatomy and the skin surface with high-fidelity, and can readily be used as a drop in replacement to con-ventional blendshape models. Given an arbitrary set of skin surface meshes of an actor and only a neutral shape with estimated skull and jaw bones, our method can recover a dense anatomical substructure which constrains every point on the facial surface. We demonstrate the usefulness of our approach in several tasks ranging from shape fitting, shape editing, and performance retargeting.

Related Animatomy: An Animator-Centric, Anatomically Inspired System for 3D Facial Modeling, Animation and Transfer · Learning an Animatable Detailed 3D Face Model from In-The-Wild Images · Neural Head Avatars from Monocular RGB Videos · PointAvatar: Deformable Point-Based Head Avatars from Videos

This work builds appearance models for biological iridescence using bird feathers as a case study, covering several distinct types of feathers with different structural coloration mechanisms in their barbules. It introduces procedural geometric models of feather barbules, a fast wave-optics simulator for computing barbule scattering, and a pipeline that produces a feather BRDF. The resulting models reproduce the angle-dependent color shifts of real iridescent plumage and won the SIGGRAPH Asia 2024 Best Paper Award.

Related Rendering Iridescent Rock Dove Neck Feathers · A Practical Extension to Microfacet Theory for the Modeling of Varying Iridescence · Microstructure-based Appearance Rendering for Feathers · A Surface-based Appearance Model for Pennaceous Feathers

Apteryx is a Weta FX feather toolset that provides a complete workflow of procedural generation, hand sculpting and grooming for feathered creatures and costumes. Developed during Kingdom of the Planet of the Apes to build a village of digital eagles, it produces simulation-ready models with realistic dynamics under motion and wind.

Related Mesh-Driven Generation and Animation of Groomed Feathers · Hummingbird: DreamWorks Feather System · A Modernization of the DreamWorks Feather System · Feathers for Mystical Creatures: Pegasus

For Walt Disney Animation Studios’ 100th anniversary feature, the filmmakers wanted to honor the studio’s legacy with a stylized look that draws from the rich artistic heritage of Disney’s earliest films. In addition to the challenges of a highly art-directed stylized look, Asha’s hairstyle comprises a full head of long, thin, tightly-braided locks which was far more complex than our previous braid grooms. In order to art direct Asha’s stylized hair performance in "Wish", several advancements were made in tools and workflows across the character departments for grooming, simulation, and stylization techniques.

Related Scriptable Character FX Solution · Choreography of Hair and Cloth in Disney's Moana 2 · Hierarchical Controls for Art-Directed Hair at Disney · The Art and Technology of Hair Simulation in Disney's Moana

Motion diffusion models and Reinforcement Learning (RL) based control for physics-based simulations have complementary strengths for human motion generation. The former is capable of generating a wide variety of motions, adhering to intuitive control such as text, while the latter offers physically plausible motion and direct interaction with the environment. In this work, we present a method that combines their respective strengths. CLoSD is a text-driven RL physics-based controller, guided by diffusion generation for various tasks. Our key insight is that motion diffusion can serve as an on-the-fly universal planner for a robust RL controller. To this end, CLoSD maintains a closed-loop interaction between two modules -- a Diffusion Planner (DiP), and a tracking controller. DiP is a fast-responding autoregressive diffusion model, controlled by textual prompts and target locations, and the controller is a simple and robust motion imitator that continuously receives motion plans from DiP and provides feedback from the environment. CLoSD is capable of seamlessly performing a sequence of different tasks, including navigation to a goal location, striking an object with a hand or foot as specified in a text prompt, sitting down, and getting up. https://guytevet.github.io/CLoSD-page/

Related Perpetual Humanoid Control for Real-time Simulated Avatars · SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation · Generating Diverse and Natural 3D Human Motions from Text · TEMOS: Generating Diverse Human Motions from Textual Descriptions

We present a new method to bake classical facial animation blendshapes into a fast linear blend skinning representation. Previous work explored skinning decomposition methods that approximate general animated meshes using a dense set of bone transformations; these optimizers typically alternate between optimizing for the bone transformations and the skinning weights. We depart from this alternating scheme and propose a new approach based on proximal algorithms, which effectively means adding a projection step to the popular Adam optimizer. This approach is very flexible and allows us to quickly experiment with various additional constraints and/or loss functions. Specifically, we depart from the classical skinning paradigms and restrict the transformation coefficients to contain only about 90% non-zeros, while achieving similar accuracy and visual quality as the state-of-the-art. The sparse storage enables our method to deliver significant savings in terms of both memory and run-time speed. We include a compact implementation of our new skinning decomposition method in PyTorch, which is easy to experiment with and modify to related problems.

Related Fast and Efficient Skinning of Animated Meshes · Robust and Accurate Skeletal Rigging from Mesh Sequences · Efficient Dynamic Skinning with Low-Rank Helper Bone Controllers · NeuroSkinning: Automatic Skin Binding for Production Characters with Deep Graph Networks

Autodesk's Lance Thornton and Evan Atherton present two AI approaches to character work in Maya. Thornton demonstrates the machine learning deformer, which trains locally on automatically generated or animation-driven poses to approximate a complex deformer or chain (such as a hero muscle system) with a single constant-time node, showing roughly 4x faster playback and interactive manipulation on a Digital Fish rig, support for bodies and faces, paintable weights and falloffs, CPU or GPU evaluation, switching back to the full rig for polish, and a mesh-compare heat map. Atherton then shows a neural motion control research prototype built in Bifrost, where a small neural network trained on under an hour of motion capture predicts each next pose from the current pose and a keyframed trajectory, producing art-directable walk, run, jump, and sit behaviors for quadrupeds and bipeds. The motion is passed through Human IK to retarget behavior across rigs of differing proportions and bake to standard controllers, with animation layers for final tweaks and Bifrost particle instancing for crowds.

Related Mobilizing Mocap, Motion Blending, and Mayhem: Rig Interoperability for Crowd Simulation on Incredibles 2 · One Model to Rig Them All: Diverse Skeleton Rigging with UniRig · Motion Retargeting for Crowd Simulation · MoRig: Motion-Aware Rigging of Character Meshes from Point Clouds

We present an overview of Animal Logic’s curve rigging toolset and its development process, serving as a case study to discuss challenges specific to doing software development for animated feature film production. We will show how R&D projects at Animal Logic lean on agile software practices to enable ambitious development projects, with flexible plans that adapt to the reality of working with creative stakeholders. We will highlight the importance of production engagement, reflect on our technical decisions made over a year of active development while reacting to drastic production schedule changes, and share lessons learned along the way.

Related USD in Production · Group Based Rigging of Realistically Feathered Wings · ChopRig System · Rubber Hose Revival: The DreamWorks Curvy-Limb Rig

Many VR animal embodiment sytsems suffer from poor animation fidelity, typically animating the animal avatars using inverse kinematics. We address this issue, presenting a novel deep-learning method, centred around a shared codebook, for mapping human motion to quadruped motion. Rather than trying to directly bridge the gap from human motion to quadruped motion, a task which has proven difficult, we first use a rule-based retargeter, relying on inverse and forward kinematics, to retarget human motions to an intermediate motion domain in which the motions share the same skeleton as the quadruped. We then use finite scalar quantization to construct a shared latent space, or codebook, between this intermediate domain and the quadruped motion domain. We do this by first pre-defining a finite number of discrete latent codes and then teaching these codes, using unsupervised deep-learning, to represent semantically similar motions in the two domains. We incorporate our real-time human-to-quadruped motion mapping into a VR quadruped embodiment system. The output quadruped animations are natural and realistic, while also preserving the semantics of users’ actions. Moreover, there is a strong synchrony between the input human motions and retargeted quadruped motions, an important factor for inducing a strong sense of VR embodiment.

Related Automated Extraction and Parameterization of Motions in Large Data Sets · Normalized Euclidean Distance Matrices for Human Motion Retargeting · Robust Marker Trajectory Repair for MOCAP Using Kinematic Reference · Physically Based Motion Transformation

Virtual garment simulation has become increasingly important with applications in garment design and virtual try-on. However, reproducing garments faithfully remains a cumbersome process. We propose an end-to-end forward pipeline for estimating parameters of shell material models corresponding to real fabrics with minimal input. In contrast to prior work that relies on complex and often expensive capture systems, our method determines yarn model parameters from Young’s moduli determined during standard yarn stretch tests. We use an extended homogenization method to match yarn-level and shell-level hyperelastic energies with respect to a range of surface deformations represented by the first and second fundamental forms, including anisotropic bending. We optimize the parameters of a shell material model involving uncoupled bending and membrane energies. This allows the simulated shell model to exhibit deformation modes motivated by yarn-level physics in real fabrics. Finally, we validate our results with quantitative and visual comparisons against real world fabrics through stretch tests and drape experiments. Using the homogenized parameters, the shell models are capable of capturing the characteristics of underlying yarn patterns and exhibiting distinct behaviors for different yarn materials.

Related Homogenized Yarn-Level Cloth · Sag-Free Initialization for Strand-Based Hybrid Hair Simulation · Interactive Hair Simulation on the GPU Using ADMM · Simulating Cloth Using Bilinear Elements

Animation is a challenging and time-consuming process where animators must manipulate hundreds of controls over space and time to create compelling motions. Recent advances in motion diffusion models have shown impressive results for general motion generation and hold the potential to reduce the number of controls manipulated by animators to achieve high quality results. However, these models are limited by their inability to match sparse constraints precisely, preventing frame-level joint control required by artists. Additionally, recent models are trained for specific characters, preventing reuse, and are incompatible for characters with only a small datasets available. To tackle these shortcomings, we propose a novel factorization of motion between a character-agnostic Bézier Motion Model (BMM), which can be trained on a large motion dataset, followed by a character-specific posing model, trainable on a much smaller pose dataset, that enables reuse across many characters. BMM provides accuracy for meeting sparse joint-level constraints by working in a reduced space of Bézier curves that better aligns the condition signal with the prediction space of our model. Additionally, the Bézier curves offer animators an intuitive interface compatible with existing authoring software.

Related SKEL-Betweener: a Neural Motion Rig for Interactive Motion Authoring · Dog Code: Human to Quadruped Embodiment Using Shared Codebooks · Improv: A System for Scripting Interactive Actors in Virtual Worlds · Character Motion Synthesis by Topology Coordinates

The paper presents a method to animate dynamic skin wrinkles under facial expressions by modifying how wrinkles are constructed in displacement maps based on local stretch and compression. Wrinkles perpendicular to the stress axis become wider when stretched and deeper when compressed. A fast approximation divides wrinkles into orientation bins with pre-computed displacement maps that are blended in a shader, achieving 50x speedup compared to the per-frame approach.

Related Cloth and Skin Deformation with a Triangle Mesh Based Convolutional Neural Network · A Neural Network Model for Efficient Musculoskeletal-Driven Skin Deformation · The Digital Emily Project: Achieving a Photorealistic Digital Actor · A Pixel-Based Framework for Data-Driven Clothing

We introduce GaussianAvatars11Project page: https://shenhanqian.github.io/gaussian-avatars, a new method to create photorealistic head avatars that are fully controllable in terms of expression, pose, and viewpoint. The core idea is a dynamic 3D representation based on 3D Gaussian splats that are rigged to a parametric morphable face model. This combination facilitates photorealistic rendering while allowing for precise animation control via the underlying parametric model, e.g., through expression transfer from a driving sequence or by manually changing the morphable model parameters. We parameterize each splat by a local coordinate frame of a triangle and optimize for explicit dis-placement offset to obtain a more accurate geometric representation. During avatar reconstruction, we jointly optimize for the morphable model parameters and Gaussian splat parameters in an end-to-end fashion. We demonstrate the animation capabilities of our photorealistic avatar in several challenging scenarios. For instance, we show reen-actments from a driving video, where our method outperforms existing works by a significant margin.

Related 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting · PointAvatar: Deformable Point-Based Head Avatars from Videos · Example-Based Facial Rigging · Face2Face: Real-Time Face Capture and Reenactment of RGB Videos

Interactive motion between multiple characters is widely utilized in games and movies. However, the method for generating interactive motions considering the character's diverse mesh shape has yet to be studied. We propose a Spatio Cooperative Transformer (SCT) to retarget the interacting motions of two characters having arbitrary mesh connectivity. SCT predicts the residual of root position and joint rotations considering the shape difference between the source and target of interacting characters. In addition, we introduce an anchor loss function for SCT to maintain the geometric distance between the interacting characters when they are retargeted. We also propose a motion augmentation method with deformation-based adaptation to prepare a source-target paired dataset with an identical mesh connectivity for training. In experiments, our method achieved higher accuracy for semantic preservation and produced less artifacts of inter-penetration between the interacting characters for unseen characters and motions than the baselines. Moreover, we conducted a user evaluation using characters with various shapes, spanning low-to-high interaction levels to prove better semantic preservation of our method compared to previous studies.

Related Contact-Aware Retargeting of Skinned Motion · Aura Mesh: Motion Retargeting to Preserve the Spatial Relationships between Skinned Characters · Normalized Euclidean Distance Matrices for Human Motion Retargeting · Dog Code: Human to Quadruped Embodiment Using Shared Codebooks

GroomCap is a multi-view hair capture method that reconstructs high-fidelity strand-level hair geometry without relying on external data priors. It introduces a neural implicit representation for the hair volume that encodes high-resolution 3D orientation and occupancy from input views, trained with a volumetric 3D orientation rendering algorithm and 2D orientation distribution supervision to avoid loss of structural information from orientation blending. Initial strands are traced within the volume and then refined through a Gaussian-based optimization using a chained Gaussian representation with direct photometric supervision from images. The pipeline produces dense, scalp-rooted strand geometries across diverse hairstyles using the same parameters, supporting re-rendering, physics-based animation, and editing.

Related Simulation-Ready Hair Capture · Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction · Structure-Aware Hair Capture · GroomGen: A High-Quality Generative Hair Model Using Hierarchical Latent Representations

HIT learns to infer the 3D location of three anatomic tissue types: subcutaneous adipose tissue, lean tissue (muscles and organs), and long bones, from an external body shape. The dataset consists of 260 female and 182 male full-body MRI scans. A learned volumetric deformation field corrects for soft-tissue changes between upright and supine MRI positions. Because HIT is parameterized by SMPL, internal structures deform plausibly when bodies are reposed or reshaped. Dataset and model are publicly available.

Related BOSS: Bones, Organs and Skin Shape Model · From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans · Hand Modeling and Simulation Using Stabilized Magnetic Resonance Imaging · SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes

CG crowds have become increasingly popular this last decade in the VFX and animation industry: formerly reserved to only a few high end studios and blockbusters, they are now widely used in TV shows or commercials. Yet, there is still one major limitation: in order to be ingested properly in crowd software, studio rigs have to comply with specific prerequisites, especially in terms of deformations. Usually only skinning, blend shapes and geometry caches are supported preventing close-up shots with facial performances on crowd characters. We envisioned two approaches to tackle this: either reverse engineer the hundreds of deformer nodes available in the major DCCs/plugins and incorporate them in our crowd package, or surf the machine learning wave to compress the deformations of a rig using a neural network architecture. Considering we could not commit 5+ man/years of development into this problem, and that we were excited to dip our toes in the machine learning pool, we went for the latter. From our first tests to a minimum viable product, we went through hopes and disappointments: we hit multiple pitfalls, took false shortcuts and dead ends before reaching our destination. With this paper, we hope to provide a valuable feedback by sharing the lessons we learnt from this experience.

Related Accurate Face Rig Approximation with Deep Differential Subspace Reconstruction · Multi-Resolution Real-Time Deep Pose-Space Deformation · Dyna: A Model of Dynamic Human Shape in Motion · Subspace Neural Physics: Fast Data-Driven Interactive Simulation

Real-time character control is an essential component for interactive experiences, with a broad range of applications, including physics simulations, video games, and virtual reality. The success of diffusion models for image synthesis has led to the use of these models for motion synthesis. However, the majority of these motion diffusion models are primarily designed for offline applications, where space-time models are used to synthesize an entire sequence of frames simultaneously with a pre-specified length. To enable real-time motion synthesis with diffusion model that allows time-varying controls, we propose A-MDM (Auto-regressive Motion Diffusion Model). Our conditional diffusion model takes an initial pose as input, and auto-regressively generates successive motion frames conditioned on the previous frame. Despite its streamlined network architecture, which uses simple MLPs, our framework is capable of generating diverse, long-horizon, and high-fidelity motion sequences. Furthermore, we introduce a suite of techniques for incorporating interactive controls into A-MDM, such as task-oriented sampling, in-painting, and hierarchical reinforcement learning (See Figure 1). These techniques enable a pre-trained A-MDM to be efficiently adapted for a variety of new downstream tasks.

Related Character Controllers Using Motion VAEs · SKEL-Betweener: a Neural Motion Rig for Interactive Motion Authoring · PDP: Physics-Based Character Animation via Diffusion Policy · CLoSD: Closing the Loop between Simulation and Diffusion for Multi-Task Character Control

Physically-based simulation is a powerful approach for 3D facial animation as the resulting deformations are governed by physical constraints, allowing to easily resolve self-collisions, respond to external forces and perform realistic anatomy edits. Today's methods are data-driven, where the actuations for finite elements are inferred from captured skin geometry. Unfortunately, these approaches have not been widely adopted due to the complexity of initializing the material space and learning the deformation model for each character separately, which often requires a skilled artist followed by lengthy network training. In this work, we aim to make physics-based facial animation more accessible by proposing a generalized physical face model that we learn from a large 3D face dataset. Once trained, our model can be quickly fit to any unseen identity and produce a ready-to-animate physical face model automatically. Fitting is as easy as providing a single 3D face scan, or even a single face image. After fitting, we offer intuitive animation controls, as well as the ability to retarget animations across characters. All the while, the resulting animations allow for physical effects like collision avoidance, gravity, paralysis, bone reshaping and more.

Related An Implicit Physical Face Model Driven by Expression and Style · Phace: Physics-based Face Modeling and Animation · BlendForces: A Dynamic Framework for Facial Animation · Shape Targeting: A Versatile Active Elasticity Constitutive Model

Marker‐based optical motion capture (MoCap) aims to localize 3D human motions from a sequence of input raw markers. It is widely used to produce physical movements for virtual characters in various games such as the role‐playing game, the fighting game, and the action‐adventure game. However, the conventional MoCap cleaning and solving process is extremely labor‐intensive, time‐consuming, and usually the most costly part of game animation production. Thus, there is a high demand for automated algorithms to replace costly manual operations and achieve accurate MoCap cleaning and solving in the game industry. In this article, we design a divide‐and‐conquer‐based MoCap solving network, dubbed MarkerNet, to estimate human skeleton motions from sequential raw markers effectively. In a nutshell, our key idea is to decompose the task of direct solving of global motion from all markers into first modeling sub‐motions of local parts from the corresponding marker subsets and then aggregating sub‐motions into a global one. In this manner, our model can effectively capture local motion patterns w.r.t. different marker subsets, thus producing more accurate results compared to the existing methods. Extensive experiments on both real and synthetic data verify the effectiveness of the proposed method.

Related Motion Retargetting based on Dilated Convolutions and Skeleton-Specific Loss Functions · Robust Marker Trajectory Repair for MOCAP Using Kinematic Reference · Motion Retargeting for Crowd Simulation · A Facial Motion Retargeting Pipeline for Appearance Agnostic 3D Characters

Crafting a single, versatile physics-based controller that can breathe life into interactive characters across a wide spectrum of scenarios represents an exciting frontier in character animation. An ideal controller should support diverse control modalities, such as sparse target keyframes, text instructions, and scene information. While previous works have proposed physically simulated, scene-aware control models, these systems have predominantly focused on developing controllers that each specializes in a narrow set of tasks and control modalities. This work presents MaskedMimic, a novel approach that formulates physics-based character control as a general motion inpainting problem. Our key insight is to train a single unified model to synthesize motions from partial (masked) motion descriptions, such as masked keyframes, objects, text descriptions, or any combination thereof. This is achieved by leveraging motion tracking data and designing a scalable training method that can effectively utilize diverse motion descriptions to produce coherent animations. Through this process, our approach learns a physics-based controller that provides an intuitive control interface without requiring tedious reward engineering for all behaviors of interest. The resulting controller supports a wide range of control modalities and enables seamless transitions between disparate tasks.

Related SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation · CLoSD: Closing the Loop between Simulation and Diffusion for Multi-Task Character Control · MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations · Generating Diverse and Natural 3D Human Motions from Text

In this work, we present MoConVQ, a novel unified framework for physics-based motion control leveraging scalable discrete representations. Building upon vector quantized variational autoencoders (VQ-VAE) and model-based reinforcement learning, our approach effectively learns motion embeddings from a large, unstructured dataset spanning tens of hours of motion examples. The resultant motion representation not only captures diverse motion skills but also offers a robust and intuitive interface for various applications. We demonstrate the versatility of MoConVQ through several applications: universal tracking control from various motion sources, interactive character control with latent motion representations using supervised learning, physics-based motion generation from natural language descriptions using the GPT framework, and, most interestingly, seamless integration with large language models (LLMs) with in-context learning to tackle complex and abstract tasks.

Related MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting · TEMOS: Generating Diverse Human Motions from Textual Descriptions · PDP: Physics-Based Character Animation via Diffusion Policy · MuscleVAE: Model-Based Controllers of Muscle-Actuated Characters

Feathers behave in a highly anisotropic way governed by their hierarchical microstructure of barbs clamped onto a rachis and linked by tiny barbules, which prior cloth-strip approximations fail to capture. Using measurement protocols on real feather samples, the authors find a linear orientation-dependent strain-stress relationship and an extreme ratio of stiffnesses between the barb and barbule directions, leading to a three-parameter anisotropic elastic shell model. They overcome the resulting numerical locking and ill-conditioning by aligning the mesh with barb directions and replacing the stiffest modes with an inextensibility constraint, then add anisotropic bending and demonstrate full-feather and bird-scale scenarios.

Related Biological Modeling of Feathers by Morphogenesis Simulation · A Biologically-Parameterized Feather Model · Animating Puss in Boots' Feather in Shrek 2 · Procedurally Generating Biologically Driven Feathers

We present a hard-real-time multi-resolution mesh shape deformation technique for skeleton-driven soft-body characters. Producing mesh deformations at multiple levels of detail is very important in many applications in computer graphics. Our work targets applications where the multi-resolution shapes must be generated at fast speeds ("hard-real-time", e.g., a few milliseconds at most and preferably under 1 millisecond), as commonly needed in computer games, virtual reality and Metaverse applications. We assume that the character mesh is driven by a skeleton, and that high-quality character shapes are available in a set of training poses originating from a high-quality (slow) rig such as volumetric FEM simulation. Our method combines multi-resolution analysis, mesh partition of unity, and neural networks, to learn the pre-skinning shape deformations in an arbitrary character pose. Combined with linear blend skinning, this makes it possible to reconstruct the training shapes, as well as interpolate and extrapolate them. Crucially, we simultaneously achieve this at hard real-time rates and at multiple mesh resolution levels. Our technique makes it possible to trade deformation quality for memory and computation speed, to accommodate the strict requirements of modern real-time systems. Furthermore, we propose memory layout and code improvements to boost computation speeds.

Related NeuroSkinning: Automatic Skin Binding for Production Characters with Deep Graph Networks · Fast and Deep Deformation Approximations · Implementing a Machine Learning Deformer for CG Crowds: Our Journey · Data-Driven Physics for Human Soft Tissue Animation