Introduction
Esports titles like League of Legends (LoL) present a significant barrier to entry for new viewers due to their fast-paced nature and complex terminology. While automated game event detection—identifying critical moments like player kills or tower takedowns—could aid accessibility, existing models often struggle. Most rely on single modalities (e.g., video or raw game data) and fail to capture the contextual "vibe" provided by excited commentators or the emotional surge in live chat.
Methodology: The 3M-Game Framework
The authors introduce 3M-Game, a framework utilizing Multi-Modal, Multi-Task, and Multi-Teacher learning. The core innovation lies in a Knowledge Distillation approach that allows a lightweight "student" model to inherit expertise from multiple specialized "teacher" models.
The framework consists of:
- Specialized Teachers: Three independent models are fine-tuned for specific domains: an Audio Teacher (AST) for sound-based action detection, a Chat Teacher (XLM-RoBERTa) for emotion tagging in live streams, and a Transcript Teacher (RoBERTa) for identifying game entities in commentary.
- Student Model: A smaller RoBERTa model trained to detect events using only text transcripts.
- Knowledge Distillation: During training, the student mimics both the "soft labels" (predictions) and "hidden states" (internal representations) of the teachers. This enables the student to leverage multimodal context—like audio cues and chat sentiment—at inference time without requiring those data streams as direct inputs.
Experimental Setup
The model was evaluated using the Game-MUG dataset, comprising League of Legends matches from Twitch and YouTube. Raw audio was converted to text using OpenAI’s Whisper. The final dataset included 28,145 instances, split 95/5 for training and testing.
Key Results and Discussion
The 3M-Game framework demonstrates a substantial performance leap over the baseline (Zhang et al. 2024).
As shown in the results table, the model achieved an overall precision of 0.495. Most notably, when focusing on "Main Events" (Kills, Towers, Dragons), the precision reached 0.638, more than tripling the baseline's 0.185.
Ablation studies highlight the importance of each modality. Removing the Chat teacher caused the most significant performance drop (from 0.495 to 0.325), suggesting that viewer sentiment is a critical indicator of game events. PCA visualizations further confirmed that the distillation process successfully organized the model's internal embeddings into distinct clusters corresponding to different event types.
Conclusion and Future Work
3M-Game proves that multi-teacher distillation can effectively "bake" multimodal knowledge into a text-only student model, significantly improving event detection in complex esports environments. Future research will focus on reducing latency for real-time applications and testing the framework's generalizability across different game genres like First-Person Shooters.
Summary updated