TL;DR
This paper introduces a novel real-time system for controlling human character locomotion styles by combining an animation synthesis network with a style modulation network. They also release a large, diverse dataset (100STYLE) to address the limitations of existing methods in handling numerous, complex motion styles in dynamic, user-controlled scenarios.
What Problem Does This Paper Address?
Creating realistic and varied human character movements in real-time animation systems is a significant challenge. Traditional style transfer methods often require a predefined motion clip as input, making them unsuitable for interactive applications where the character's future movements are unknown and can change rapidly based on user input. Existing animation synthesis systems, while good at generating motion, often struggle to model a wide variety of distinct styles or require extensive retraining for new styles. This paper aims to bridge this gap by enabling real-time, style-controllable locomotion that can adapt to a large number of diverse styles without constant retraining.
Methods
The core of the proposed system is a two-part neural network architecture:
- Animation Synthesis Network: This network is based on a "gated experts" approach, similar to existing methods, which predicts future motion frames. A key innovation here is the use of local motion phases instead of global ones. This is crucial because many stylized motions lack a clear, synchronized global phase (e.g., when arms and legs move independently). To handle motions where foot contact information might be unreliable or absent, they introduce a contact-free local phase formulation. This allows the system to extract phase information from joint positions, making it more robust.
- Style Modulation Network: This network takes a motion clip as input and learns a compact style representation. It uses Feature-Wise Linear Modulation (FiLM) to transform the hidden layers of the animation synthesis network. FiLM is powerful because it can both scale and shift features, allowing for more expressive style control compared to methods that only shift features (like residual adapters) or use simple one-hot encodings. This network learns to parameterize style as FiLM parameters, which are then applied to the animation synthesis network.
The system combines these two components: the animation synthesis network generates motion based on local phases, and the style modulation network injects style information via FiLM parameters.
Datasets
- 100STYLE Dataset: The authors introduce a new dataset called 100STYLE, containing over 4 million frames of motion capture data across 100 different locomotion styles. This dataset is designed to be challenging, featuring styles that are stochastic, asymmetric, and lack clear global phases, aiming to push the boundaries of existing style modeling techniques. It includes various gaits like walking, running, sidestepping, and transitions.
Key Insights and Ideas
- Local Phases are Crucial for Diverse Styles: The paper strongly advocates for using local motion phases over global ones, especially for stylized locomotion. This allows the system to capture the independent timing and movement of different body parts, which is essential for complex and non-uniform gaits.
- Contact-Free Local Phases Broaden Applicability: The development of a contact-free method for calculating local phases is a significant contribution. It makes the system applicable to a wider range of motions where precise foot or hand contact information might be unavailable or unreliable.
- FiLM Offers Superior Style Representation: The use of FiLM for style modulation proves to be more effective than one-hot encoding or residual adapters. FiLM's ability to scale and shift features allows for richer style representation, smoother interpolations between styles, and better handling of complex motions.
- Compact Style Representation Enables Efficiency: The FiLM parameters learned for each style are compact and can be pre-computed. This allows for efficient storage and real-time application, and importantly, enables easy fine-tuning for new styles without retraining the entire animation synthesis network.
Key Results
- Improved Motion Quality: The proposed system demonstrates improved motion quality, particularly in handling complex styles that challenge existing methods.
- Robustness and Efficiency: The system is shown to be more robust and efficient in its style representation compared to other real-time style modeling approaches.
- Smooth Interpolation: FiLM-based style modulation allows for smooth, continuous interpolation between different locomotion styles, a capability lacking in one-hot encoded methods.
- Effective New Style Modeling: The compact FiLM representation allows for efficient fine-tuning to model new styles, a significant advantage over retraining entire networks.
- Quantitative Performance: While specific quantitative metrics like MSE are discussed, the paper emphasizes that qualitative performance and robustness to failure cases are more critical than raw error reduction. They show their method performs competitively or better on various metrics and qualitative evaluations.
Limitations and Future Work
- Stochasticity Handling: The current contact-free local phase formulation does not handle stochastic motions particularly well. Future work could focus on designing source functions that better capture such variations.
- Physics Inconsistency: While the system interpolates smoothly, it can sometimes produce physically implausible poses or "clipping" when transitioning between styles. The authors suggest incorporating physics understanding or skeletal awareness into future models to address this.
- Subjectivity of Style: The paper acknowledges that "style" is a subjective concept, and their definition of style (what differs between motions) is based on the data and model's assumptions. Future work could explore more creative or user-defined style generation.
Summary updated