Abstract
Recently, in the field of computer animation, a method for generating motion using deep learning has been studied away from conventional finite-state machines or graph-based methods. The expressiveness of the network required for learning motions is more influenced by the diversity of motion contained in it than by the simple length of motion to be learned. This study aims to find an efficient network structure when the types of motions to be learned are diverse. In this paper, we train and compare three types of networks: basic fully-connected structure, mixture of experts structure that uses multiple fully-connected layers in parallel, recurrent neural network which is widely used to deal with seq2seq, and transformer structure used for sequence-type data processing in the natural language processing field.
Introduction
In modern AAA games, character animation quality is paramount. Traditionally, developers used Finite State Machines (FSMs) or Motion Matching. However, as games grow in volume and diversity, these methods become unmanageable; FSMs require exhaustive manual labor by artists, and Motion Matching demands massive amounts of memory to store every possible animation clip.
Deep learning offers a solution by "compressing" these motions into neural networks. However, a network’s ability to learn is constrained not just by the length of the animation data, but by its diversity. This research tackles the specific challenge of hit reactions—motions that vary wildly depending on the force of the impact, the direction of the attack, and which body part was hit. The goal is to identify which neural network architecture is most efficient at capturing this high-dimensional variety.
Methodology
The researchers implemented a system that generates a "hit reaction" in two distinct phases: a short physics-based simulation (using the DART library) for the immediate impact, followed by a neural network that predicts the subsequent recovery motion to return the character to an idle state.
Dataset Generation
Because no existing dataset captured the specific nuances of hit reactions needed, the authors created a custom synthetic dataset using physics simulation. The dataset consists of approximately 8 hours of motion data (830,000 frames at 30Hz). Variables include randomly varied hit intensity (300N to 900N), attack direction ($-\pi$ to $\pi$), and hit locations across the upper body. Data was converted into pose sequences including root position, joint rotations, foot contact info, and control parameters.
Architectural Comparison
The study compared four distinct architectures to determine which handles motion diversity most effectively:
- Fully-Connected (FC): The baseline, tested with 1024 and 2048 nodes per layer.
- Mixture of Experts (MoE): A structure where multiple "expert" networks work in parallel, governed by a "gating" network that decides which expert's output is most relevant to the current input.
- Recurrent Neural Network (RNN): Specifically using Long Short-Term Memory (LSTM) units, designed to handle sequential data by maintaining an internal "memory" state.
- Transformer: A self-attention-based model (encoder-only) designed to weigh the importance of different past frames to predict the next one.
To ensure realism, they utilized LSGAN (Least Squares Generative Adversarial Network) as an additional loss term, forcing the networks to produce motions indistinguishable from real data.
Results and Discussion
Quantitative comparisons across the models yielded clear winners regarding efficiency and accuracy:
- The MoE Advantage: The Mixture of Experts model excelled because it allows different sub-networks to specialize in different types of reactions. This "divide and conquer" approach is more efficient than forcing a single large FC network to learn everything. The MoE model achieved a computation time of 1.39ms with 8.47M parameters.
- Accuracy vs. Weight: The FC-2048 model had the lowest loss (most accurate) but was the heaviest, with 16.12M parameters and a slower 3.04ms computation time. The MoE model showed a learning curve nearly identical to the much larger FC-2048 model but with roughly half the parameters and twice the speed.
- Computational Overhead: Interestingly, while Transformers and RNNs are mathematically sophisticated for sequences, they introduced significant computational overhead. The Transformer had the fewest parameters but was slower than the MoE and FC-1024 models due to the complexity of attention mechanism operations. The RNN was the slowest to learn and had the highest final loss.
Conclusion
The study confirms that the expressiveness of a network is tested more by the variety of motions than the sheer number of frames. While all models eventually learned to produce natural motions, the Mixture of Experts (MoE) structure offers the best balance between learning accuracy, parameter efficiency, and real-time computational speed for complex tasks like hit reactions. Future work may involve fine-tuning these architectures or developing objective evaluation standards for "human-like" motion quality.
Summary updated