In-Game Toxic Language Detection: Shared Task and Attention Residuals (Student Abstract)

Yuanzhe Jia, Weixuan Wu, Feiqi Cao, Soyeon Caren Han

Abstract

Detecting toxic language in online gaming environments presents unique challenges due to extreme brevity, niche jargon, and the need for fine-grained token-level identification. This paper introduces BRAR (Bi-directional Representations with Attention Residuals), a novel architecture designed for the CONDA shared task. By integrating Bi-LSTM layers with an Attention Residual mechanism and a Label Forcing technique, BRAR achieves a state-of-the-art 99.9% Micro F1-score on the CONDA dataset. Our findings suggest that shallow, context-aware models are highly effective for the short, specialized sequences found in in-game chat.

Introduction

Toxic behavior remains a significant barrier to positive player experiences in online gaming. However, standard Natural Language Processing (NLP) models often struggle with the idiosyncratic nature of in-game communication. Unlike formal text or even general social media posts, game chat is characterized by:

  1. Extreme Brevity: Players communicate in short bursts to avoid interrupting gameplay.
  2. Niche Context: Utterances are saturated with game-specific acronyms, character names, and slang.
  3. Slot-Level Complexity: Toxicity is often localized to specific tokens rather than the entire sentence.

To address these issues, we focus on token-level detection (slot filling), aiming to categorize each word into classes such as Toxicity (T), Game Slang (S), or Dota-specific terms (D).

Methodology: The BRAR Architecture

The proposed BRAR model (Bi-directional Representations with Attention Residuals) is designed to balance local token dependencies with global utterance context.

The BRAR architecture featuring Bi-LSTM, Attention Residuals, and Label Forcing layers.

Core Components

  • Bi-LSTM & CRF: The model uses a Bi-directional Long Short-Term Memory (Bi-LSTM) network to capture sequential patterns. A Conditional Random Field (CRF) layer follows to ensure that the predicted sequence of labels is structurally coherent.
  • Attention Residuals: This novel mechanism extracts global information from the entire utterance. This global context is treated as a "residual" and added back to the local token representations. This allows the model to disambiguate words based on the overall intent of the message.
  • Label Forcing (LF): Recognizing the highly repetitive nature of gaming vocabulary (e.g., "gg"), the LF layer calculates token probability distributions from the training corpus. It provides a strong prior for frequently occurring terms, maintaining consistency during inference.
  • Embeddings: To handle typos and out-of-vocabulary (OOV) words, the model employs FastText-50 embeddings, which leverage sub-word information.

Experimental Setup

The model was evaluated using the CONDA dataset, which consists of 44,869 utterances from over 1,900 Dota 2 matches. The task involves labeling each token into one of six categories: Toxicity (T), Character (C), Dota-specific (D), Game Slang (S), Pronoun (P), and Other (O).

Results and Discussion

BRAR significantly outperformed existing benchmarks on the CONDA dataset, setting a new state-of-the-art performance level.

Comparison of BRAR against baseline models on the CONDA dataset.

Performance Analysis

As shown in the results table, BRAR achieved a 99.9% Micro F1-score, surpassing previous high-performers like Capsule NN (99.1%) and Joint BERT (98.9%). The model demonstrated exceptional precision in identifying Toxicity (98.6% F1) and Game Slang (99.4% F1).

The Impact of Model Depth

An ablation study revealed an interesting insight: a single-layer Bi-LSTM outperformed deeper stacks (2 or 3 layers). For the extremely short sequences typical of game chat, deeper architectures appear to introduce noise or lead to overfitting, whereas a shallow, efficient model captures the signal more effectively. This efficiency makes BRAR a strong candidate for real-time moderation systems.

Conclusion

The BRAR model demonstrates that combining global attention residuals with statistical priors (Label Forcing) is a powerful strategy for specialized NLP tasks like in-game chat moderation. While the current results are specific to Dota 2, the architecture provides a robust framework for fine-grained toxicity detection. Future work should investigate the model's generalizability across different gaming genres and its resilience against adversarial text manipulation.

Summary updated

Abstract

In-game toxic language becomes the hot potato in the gaming industry and community. There have been several online game toxicity analysis frameworks and models proposed. However, it is still challenging to detect toxicity due to the nature of in-game chat, which has extremely short length. In this paper, we describe how the in-game toxic language shared task has been established using the real-world in-game chat data. In addition, we propose and introduce the model/framework for toxic language token tagging (slot filling) from the in-game chat. The data and code will be released.
BibTeX
@inproceedings{jia2023game,
  author = {Yuanzhe Jia and Weixuan Wu and Feiqi Cao and Soyeon Caren Han},
  title = {{In-Game Toxic Language Detection: Shared Task and Attention Residuals (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i13.26979},
}
Figure