A K-Competitive Autoencoder for Aggression Detection in Social Media Text

Promita Maitra, Ritesh SarkhelTRAC @ COLING 2018 · OpenReview

Abstract

Detecting aggression in social media text is a complex task due to linguistic noise, context sparsity, and feature redundancy. This paper introduces Emoti-KATE, a shallow autoencoder that utilizes a "winner-takes-all" K-competitive layer to refine document representations. By forcing hidden neurons to compete and reallocating energy to the most active units, the model effectively de-noises sparse data. Experimental results on the TRAC 2018 dataset demonstrate that Emoti-KATE significantly outperforms traditional baselines like Word2vec and LDA, particularly in identifying both overt and covert aggression in English and Hindi social media posts.

Introduction

Aggression detection on platforms like Facebook and Twitter is hindered by the nature of social media communication: posts are typically short, informal, and laden with sarcasm or typos. These characteristics lead to two primary challenges for Natural Language Processing (NLP) models: context sparsity, where limited text provides few features for classification, and feature redundancy, where standard neural architectures may learn overlapping or noisy patterns that fail to distinguish subtle aggressive intent.

To address these issues, this research proposes a robust document representation method designed to handle the linguistic chaos of multi-lingual (English and Hindi) and code-mixed text. The goal is to create a model that filters out redundant information to focus on the most salient indicators of aggression.

Methodology

The Emoti-KATE Architecture

The core innovation of this work is the Emoti-KATE autoencoder. Unlike standard autoencoders where all hidden neurons contribute to reconstruction, Emoti-KATE implements a K-competitive layer to simulate competition among features.

The Emoti-KATE architecture, highlighting the K-competitive hidden layer where energy from 'loser' neurons is reallocated to 'winners'.

During the feedforward process, the model identifies the top $K/2$ neurons with the highest positive activations and the $K/2$ neurons with the lowest negative activations. The remaining "loser" neurons are zeroed out, and their activation energy is reallocated to the winners using an amplification factor ($\alpha$). This mechanism forces the network to learn only the most unique and discriminative patterns, effectively de-noising the representation.

Feature Engineering and Datasets

The model was evaluated using three distinct levels of input feature engineering to capture different linguistic nuances:

Description of the three input feature vectors (X1, X2, X3) used to evaluate the model's performance.

The system was tested on the TRAC 2018 workshop dataset, comprising 15,000 aggression-annotated posts across four subsets: English-Facebook, English-Twitter, Hindi-Facebook, and Hindi-Twitter. The task involved a 3-way classification: Overtly Aggressive (OAG), Covertly Aggressive (CAG), and Non-aggressive (NAG).

Results and Discussion

Emoti-KATE demonstrated a marked improvement over traditional methods, achieving a 19.47% average improvement over standard variational autoencoders. Across all datasets, the model outperformed Word2vec by 13.62% and LDA-based representations by 16.25%.

Key findings include:

  • Hashtag Utility: For English Facebook data, the $X_3$ vector (segmenting hashtags) yielded the best results, suggesting that hashtags often contain the primary sentiment in short-form text.
  • Covert Aggression: The model proved surprisingly effective at identifying "Covertly Aggressive" text—sentences that lack explicit hate words but carry aggressive intent. This is attributed to the competitive layer's ability to isolate subtle, non-redundant patterns.
  • Cross-Lingual Challenges: Performance on Hindi text (F1 ~0.41) was lower than English (F1 ~0.57), largely due to the lack of high-quality sentiment lexicons for code-mixed Hindi and the failure of English-centric hashtag segmentation on Romanized Hindi scripts.

Conclusion

Emoti-KATE provides a powerful framework for aggression detection by leveraging competition to enhance feature clarity. While the model struggles with out-of-vocabulary terms and complex sarcasm, it represents a significant step forward in processing noisy social media data. Future work will explore competitive stacked denoising autoencoders and the development of more robust, code-mixed corpora to improve performance in multilingual contexts.

Summary updated

BibTeX
@inproceedings{maitra2018k,
  author = {Promita Maitra and Ritesh Sarkhel},
  title = {{A K-Competitive Autoencoder for Aggression Detection in Social Media Text}},
  booktitle = {TRAC @ COLING},
  year = {2018},
  url = {https://openreview.net/forum?id=Qwy99-MByD},
}
Figure