TL;DR
This paper introduces a neural framework that automatically rigs and skins 3D character meshes of any topology, producing high-quality animations compatible with standard motion capture (mocap) data. By combining a base skeletal deformation model with "neural blend shapes"—learned corrective displacements—the system eliminates common artifacts like "elbow collapse" without requiring ground-truth rigging data for training.
What Problem Does This Paper Address?
In the world of 3D animation, bringing a static character mesh to life is a grueling process. Animators must first "rig" the character (create a skeleton) and then "skin" it (define how much each bone influences specific parts of the mesh). Standard techniques like Linear Blend Skinning (LBS) are computationally efficient but prone to ugly artifacts, such as joints losing volume when bent (the "elbow collapse" or "candy wrapper" effect).
While recent deep learning methods have tried to automate this, they often struggle with two things:
- Connectivity Constraints: Many models only work on meshes with a fixed number of vertices and specific edge connections.
- Data Scarcity: High-quality rigging and skinning data is hard to come by. Most existing models require direct supervision from artist-created rigs, which are expensive to produce.
This paper aims to create a system that works on arbitrary meshes, produces mocap-compatible rigs, and learns how to deform them simply by looking at the final moving shapes, rather than the underlying rig parameters.
Methods
The authors propose a two-branch neural architecture that mimics the classical animation pipeline:
- Envelope Deformation Branch: This branch handles the "base" movement. It uses MeshCNN to extract features from the input mesh surface and a skeleton-aware network to predict joint offsets (the rig) and skinning weights. Crucially, the skeletal hierarchy is pre-defined to match common mocap formats, ensuring the output is immediately usable in game engines.
- Residual Deformation Branch: This is the "secret sauce." It predicts neural blend shapes—pose-dependent displacements that correct the errors of the base envelope. As the character moves, a small MLP calculates blending coefficients based on joint rotations, adding fine-grained detail and volume preservation to joint regions.
A standout feature is Indirect Supervision. Instead of being told "this vertex should have a weight of 0.5 for the arm bone," the network is trained by comparing its final articulated mesh to a ground-truth deformed shape. It "back-calculates" the best rigging and skinning parameters to achieve that final look.
Datasets
The model was trained and evaluated using a mix of synthetic and real-world data:
- SMPL Dataset: A standard parametric human body model used to generate thousands of diverse body shapes and poses.
- Multi-Garment Network (MGN): Used to provide data on clothed humans, helping the network learn to handle geometric variability like loose shirts or pants.
- Mixamo: A library of professional character rigs used to test the model's ability to generalize to "in the wild" characters created by 3D artists.
- Mesh Augmentation: The authors applied topological operations (edge flips, splits, collapses) to the training meshes to ensure the network remains robust regardless of how a mesh is triangulated.
Key Insights and Ideas
- Rigging as Attention-Based Pooling: The authors treat the rigging process (finding joint positions) as a form of attention-based pooling. Joint positions are inferred by looking at the deep features of the mesh vertices that are most strongly "bound" to them via skinning weights.
- The Power of Indirect Supervision: By supervising only the final vertex positions, the network is free to find the most efficient rigging and skinning parameters for a given topology, often outperforming methods trained on direct artist supervision.
- Generalization through Augmentation: Training on "broken" or augmented meshes (edge collapses/splits) is what allows the network to handle a high-resolution professional character just as easily as a low-poly game asset.
Key Results
The framework demonstrates significant improvements over state-of-the-art methods like RigNet and Pinocchio:
- Rigging Accuracy: In quantitative tests (measuring distances between predicted and ground-truth joints), this method achieved a Chamfer Distance (CD-J2J) of 0.012, compared to RigNet's 0.194 and Pinocchio's 0.474.
- Deformation Quality: The neural blend shapes effectively eliminate volume loss. As seen in Figure 1, the joints look natural even in extreme poses.
- Plug-and-Play: Because the output conforms to standard LBS/DQS models with corrective blend shapes, the rigs can be imported directly into software like Maya, Blender, or Unreal Engine.
Limitations and Future Work
- Fixed Hierarchy: The current model requires a pre-defined skeletal structure (e.g., a standard biped). To rig a quadruped or a creature with extra limbs, the network would need to be retrained.
- Static Deformations: The model focuses on pose-dependent corrections but does not yet account for secondary dynamics—the "jiggle" of muscles or the sway of clothing that happens due to momentum.
- Input Requirements: The system assumes the input character is in a T-pose and has a consistent upright orientation. Relaxing these requirements is a goal for future research.
Summary updated