TL;DR
This paper introduces ESARM, a novel framework for generating emotionally rich 3D facial animations from speech. By combining a speech-to-animation (STA) model with a reinforcement learning approach guided by a learned reward model, ESARM generates more diverse and human-aligned emotional expressions than previous methods.
What Problem Does This Paper Address?
Creating realistic and emotionally resonant 3D facial animations from speech is a challenging task. Existing speech-to-animation (STA) models often excel at lip synchronization but struggle to capture the subtle nuances of human emotion, leading to animations that feel robotic or fall into the "uncanny valley." This lack of emotional depth and variety fails to meet the expectations of applications in virtual reality, gaming, and film. The core problem is disentangling emotional expression from the content of speech and generating animations that not only match the spoken words but also convey the intended emotion in a way that humans find natural and appealing.
Methods
ESARM employs a two-stage approach:
- Speech-to-Animation (STA) Model: This module is designed to generate the initial facial animation. It uses a pre-trained model to extract high-level acoustic features from speech. Crucially, it incorporates a "style vector" derived from conditioning variables like emotion, intensity, and speaker identity. This allows for explicit control over the emotional tone of the animation. The model then predicts 52 blendshape coefficients, which are fundamental for driving realistic facial animation, creating an end-to-end system.
- Reward Model and Reinforcement Learning (RL): To overcome the limitations of purely supervised learning and align animations with human preferences, ESARM introduces a reinforcement learning framework.
- Pre-training: An initial STA model (GSTA) is trained on expert data.
- Automatically-Ranked Demonstrations: The GSTA model is used to generate multiple animation sequences. By systematically injecting varying levels of noise into the generation process, the resulting animations are automatically ranked based on their perceived quality (less noise generally means higher quality). This creates a dataset of ranked trajectories.
- Reward Model Training: A reward model (RM) is trained on these ranked demonstrations using a pairwise ranking loss. This RM learns to assess the quality of generated facial animations, effectively capturing human preferences.
- RL Fine-tuning: The trained RM provides a sparse reward signal to guide the STA agent (the STA model) through reinforcement learning. The agent is trained to maximize this reward, encouraging it to explore the animation space and generate sequences that are not only lip-synchronized but also emotionally expressive and aligned with human preferences.
Datasets
- MEAD Dataset: Used for training and evaluation. This dataset contains 48 individuals reciting sentences with varying emotional expressions and intensities. The authors leverage facial blend shapes from this dataset to reconstruct realistic 3D faces.
- RAVDESS Dataset: Used for evaluation.
Key Insights and Ideas
- Decoupling Emotion and Content: The framework explicitly separates the extraction of emotional cues from the speech content, allowing for more targeted emotional expression in the animation.
- Leveraging Human Preferences via Reward Modeling: Instead of relying solely on ground truth data, ESARM learns what constitutes a "good" animation by training a reward model on automatically ranked demonstrations. This allows the system to capture subjective human aesthetic judgments.
- Reinforcement Learning for Exploration: RL is used to push the animation agent beyond simply mimicking existing data, enabling it to discover novel and more expressive animation strategies that better align with human preferences.
- Automatic Ranking of Demonstrations: The clever use of noise injection to automatically rank generated animations simplifies the data collection process for reward model training, avoiding the need for manual annotation.
Key Results
- Improved Lip Synchronization and Emotional Expression: ESARM achieves superior performance in both lip synchronization (measured by Lip Vertex Error - LVE) and emotional expression (measured by Emotional Vertex Error - EVE) compared to existing methods like FaceFormer and EmoTalk. For example, on the MEAD dataset, ESARM achieved an LVE of 2.485 and an EVE of 2.982.
- Generation of Human-Preferred Animations: Qualitative comparisons show that the RL fine-tuned agent can generate complex and emotionally expressive actions that are present in human datasets but not captured by the initial STA model alone.
- Impact of Noise Level: Experiments show a clear inverse correlation between noise level and animation quality, validating the approach for generating ranked demonstrations.
Limitations and Future Work
- Limitations: The paper notes that the STA model's objective (predicting the next action code) differs from human preference for animated sequences, which is why RL fine-tuning is necessary. The agent does not have access to human datasets during RL fine-tuning, relying solely on the learned reward model.
- Future Work: Not explicitly detailed, but the success in generating more human-aligned emotional expressions suggests further exploration in fine-grained emotional control and more complex narrative-driven animations. The framework's ability to leverage speaker identity also opens avenues for personalized animation.
Summary updated