Introduction
Synthesizing high-quality 3D mesh animations is a critical task for the film and gaming industries. However, deep learning approaches face significant hurdles due to the irregular topology of 3D meshes and the high dimensionality of vertex data. Traditional methods often suffer from error accumulation or "freezing," where the model fails to sustain motion over long sequences and settles into a static average shape. This paper presents a novel framework that integrates mesh-based Convolutional Neural Networks (CNNs) with weight-shared Bidirectional LSTMs (BD-LSTMs) to generate stable, realistic 3D animations directly from mesh deformation representations.
Methodology
The proposed architecture addresses spatial geometry and temporal dynamics through a multi-stage process designed to handle the complexities of 3D data.
Spatial Encoding with Mesh CNN
To handle irregular mesh topologies, the framework uses a specialized mesh convolution layer. Instead of raw coordinates, it operates on a shape deformation representation—rotation and scaling matrices derived from deformation gradients. This encoding step reduces high-dimensional mesh data into a compact latent space ($z$), preserving local geometric relationships while making the data more manageable for temporal modeling.
Temporal Modeling with BD-LSTM
The core of the system is a weight-shared Bidirectional LSTM. Unlike standard unidirectional models that only look forward, this architecture processes sequences in both forward and backward directions simultaneously. By sharing weights and employing a bidirectional loss, the model learns a robust motion representation that minimizes drift and stabilizes long-term synthesis.
Experimental Results
The model was evaluated on diverse datasets, including Dyna (human motion) and Horse (galloping sequences), to test its ability to handle both humanoid and non-humanoid topologies.
Qualitative Performance
The framework demonstrates a remarkable ability to maintain motion cycles without "melting" or deforming the mesh unnaturally over time. It can generate long-term sequences (15+ frames) and perform "in-betweening" to complete missing segments between keyframes. Notably, the model learns the periodicity of motion; for example, in a galloping horse, it understands that the legs must return to their starting positions.
Quantitative Analysis
In per-vertex position error tests, the BD-LSTM approach significantly outperformed baseline models. On the "Punching" dataset, the proposed method achieved an error of $88 \times 10^{-4}$, compared to $114 \times 10^{-4}$ for unidirectional LSTMs. The inclusion of KL divergence loss and $L_2$ regularization was found to be crucial in preventing the model from settling into a static "average" shape, ensuring the animation remains dynamic.
Key Insights
- Stability through Bidirectionality: The bidirectional structure forces the network to maintain consistency across the entire sequence, effectively curbing the error accumulation typical of autoregressive models.
- Bootstrap Power: The model’s accuracy improves as more "bootstrap" (initial) frames are provided, allowing it to better perceive the intended velocity and direction of motion.
- Diverse Completion: By assigning a random vector to the LSTM state during sequence completion, the model can generate multiple plausible action paths from a single starting point, rather than just the shortest mathematical path.
Conclusion
This work demonstrates that combining mesh-specific spatial encodings with bidirectional temporal modeling provides a powerful tool for 3D animation synthesis. While currently limited to fixed-topology meshes, the framework sets a new standard for stability and realism in mesh-based sequence generation. Future work may explore integration with Generative Adversarial Networks (GANs) to further enhance the diversity and detail of synthesized motions.
Summary updated