Abstract
Detecting toxic language in online gaming environments presents unique challenges due to extreme brevity, niche jargon, and the need for fine-grained token-level identification. This paper introduces BRAR (Bi-directional Representations with Attention Residuals), a novel architecture designed for the CONDA shared task. By integrating Bi-LSTM layers with an Attention Residual mechanism and a Label Forcing technique, BRAR achieves a state-of-the-art 99.9% Micro F1-score on the CONDA dataset. Our findings suggest that shallow, context-aware models are highly effective for the short, specialized sequences found in in-game chat.
Introduction
Toxic behavior remains a significant barrier to positive player experiences in online gaming. However, standard Natural Language Processing (NLP) models often struggle with the idiosyncratic nature of in-game communication. Unlike formal text or even general social media posts, game chat is characterized by:
- Extreme Brevity: Players communicate in short bursts to avoid interrupting gameplay.
- Niche Context: Utterances are saturated with game-specific acronyms, character names, and slang.
- Slot-Level Complexity: Toxicity is often localized to specific tokens rather than the entire sentence.
To address these issues, we focus on token-level detection (slot filling), aiming to categorize each word into classes such as Toxicity (T), Game Slang (S), or Dota-specific terms (D).
Methodology: The BRAR Architecture
The proposed BRAR model (Bi-directional Representations with Attention Residuals) is designed to balance local token dependencies with global utterance context.
Core Components
- Bi-LSTM & CRF: The model uses a Bi-directional Long Short-Term Memory (Bi-LSTM) network to capture sequential patterns. A Conditional Random Field (CRF) layer follows to ensure that the predicted sequence of labels is structurally coherent.
- Attention Residuals: This novel mechanism extracts global information from the entire utterance. This global context is treated as a "residual" and added back to the local token representations. This allows the model to disambiguate words based on the overall intent of the message.
- Label Forcing (LF): Recognizing the highly repetitive nature of gaming vocabulary (e.g., "gg"), the LF layer calculates token probability distributions from the training corpus. It provides a strong prior for frequently occurring terms, maintaining consistency during inference.
- Embeddings: To handle typos and out-of-vocabulary (OOV) words, the model employs FastText-50 embeddings, which leverage sub-word information.
Experimental Setup
The model was evaluated using the CONDA dataset, which consists of 44,869 utterances from over 1,900 Dota 2 matches. The task involves labeling each token into one of six categories: Toxicity (T), Character (C), Dota-specific (D), Game Slang (S), Pronoun (P), and Other (O).
Results and Discussion
BRAR significantly outperformed existing benchmarks on the CONDA dataset, setting a new state-of-the-art performance level.
Performance Analysis
As shown in the results table, BRAR achieved a 99.9% Micro F1-score, surpassing previous high-performers like Capsule NN (99.1%) and Joint BERT (98.9%). The model demonstrated exceptional precision in identifying Toxicity (98.6% F1) and Game Slang (99.4% F1).
The Impact of Model Depth
An ablation study revealed an interesting insight: a single-layer Bi-LSTM outperformed deeper stacks (2 or 3 layers). For the extremely short sequences typical of game chat, deeper architectures appear to introduce noise or lead to overfitting, whereas a shallow, efficient model captures the signal more effectively. This efficiency makes BRAR a strong candidate for real-time moderation systems.
Conclusion
The BRAR model demonstrates that combining global attention residuals with statistical priors (Label Forcing) is a powerful strategy for specialized NLP tasks like in-game chat moderation. While the current results are specific to Dota 2, the architecture provides a robust framework for fine-grained toxicity detection. Future work should investigate the model's generalizability across different gaming genres and its resilience against adversarial text manipulation.
Summary updated