An Integrated Model for Social Media Toxic Comments Detection: Fusion of High-Dimensional Neural Network Representations and Multiple Traditional Machine Learning Algorithms

Jiahuai Ma, Kaixian Xu, Yu Qiao, Zhaoyan ZhangJournal of Computational Methods in Engineering Applications 2022 · DOI

Abstract

Social media platforms have become pivotal for global communication and information exchange but are increasingly challenged by the proliferation of toxic comments. These comments, characterized by abusive, discriminatory, or harassing language, threaten user safety and well-being, necessitating efficient detection systems. This paper proposes a novel hybrid approach to detect social media toxic comments by combining the feature extraction capabilities of Long Short Term Memory (LSTM)-based neural networks with multiple machine learning models, including Random Forest, Logistic Regression, and K-Nearest Neighbors. High-dimensional feature representations from the neural network are integrated with predictions from traditional classifiers, and Random Forest optimizes the output weights to maximize performance. Evaluated on a Kaggle dataset, the proposed model achieves an accuracy of 89.78% and outperforms individual models in handling the complexity of toxic comments. However, challenges such as overfitting, computational overhead, and interpretability remain. Future work aims to address these limitations through improved data augmentation, explainability methods, and more scalable architectures.

Introduction

Social media moderation is a complex challenge involving both scale and linguistic nuance. While human moderators are adept at identifying sarcasm or subtle harassment, they cannot keep pace with the millions of posts generated daily. Automated systems typically fall into two categories: rule-based filters, which are easily bypassed by slang or creative spelling, and single-model AI systems. While deep learning models like LSTMs are powerful, they often overfit on small datasets or struggle to generalize across the diverse, messy landscape of internet discourse. This research addresses these bottlenecks by proposing a hybrid "fusion" model that captures complex linguistic patterns while maintaining the stability of traditional machine learning.

Methodology

The proposed system utilizes a multi-stage pipeline where different algorithms function as specialized components. The architecture treats the neural network as a sophisticated feature generator rather than the sole decision-maker.

The proposed hybrid model architecture, integrating LSTM feature extraction with a multi-classifier fusion layer.
  1. Preprocessing & Balancing: Text is vectorized using TF-IDF (Term Frequency-Inverse Document Frequency). To address the inherent class imbalance where toxic comments are significantly less frequent than non-toxic ones, the researchers applied SMOTE (Synthetic Minority Over-sampling Technique).
  2. Feature Extraction: A Bidirectional LSTM neural network processes the text to identify long-range dependencies and context. High-level internal representations are extracted from the network to serve as features for the subsequent stage.
  3. The Committee Layer: These high-dimensional features are fed into three classic classifiers: Random Forest (RF), K-Nearest Neighbors (KNN), and Logistic Regression (LR).
  4. Model Fusion: A second Random Forest model acts as a meta-classifier ("The Judge"). It integrates the predictions from the committee and the LSTM's features to produce the final classification.

Dataset and Preprocessing

The study utilized the Kaggle Toxic Comment Classification Challenge dataset, containing over 150,000 samples. For this specific implementation, a subset of 10,000 samples was used, split into training (60%), testing (30%), and validation (10%) sets.

Initial distribution of toxicity in the dataset, illustrating the significant class imbalance.

The initial dataset exhibited a heavy skew toward non-toxic comments, necessitating the use of SMOTE to prevent the model from developing a bias toward the majority class.

Key Results

The hybrid fusion model demonstrated superior performance compared to standalone architectures:

  • Accuracy: The integrated model achieved 89.78%, a notable improvement over the standalone LSTM's 85.93%.
  • Precision and Recall: The model reached a Precision of 0.2677 and a Recall of 0.1688.
  • F1-Score: At 0.2070, the F1-score was the highest among tested configurations, indicating a more effective balance between sensitivity and specificity than individual models.
  • Optimization: Performance for the final Random Forest "Judge" peaked at approximately 200 estimators, after which accuracy gains plateaued.

Discussion and Future Work

While the fusion approach provides a performance boost, several limitations persist. The computational overhead of training both a deep neural network and multiple traditional models is significant, potentially hindering real-time scalability. Furthermore, the complexity of the fusion process introduces a "black box" problem, making it difficult to interpret the specific reasons behind a classification.

Future research will focus on:

  • Data Augmentation: Techniques to further improve the F1-score.
  • Scalability: Exploring lighter architectures for high-volume platforms.
  • Explainability: Implementing methods to provide human moderators with insights into the AI's decision-making process.

Summary updated

Abstract

Social media platforms have become pivotal for global communication and information exchange but are increasingly challenged by the proliferation of toxic comments. These comments, characterized by abusive, discriminatory, or harassing language, threaten user safety and well-being, necessitating efficient detection systems. This paper proposes a novel hybrid approach to detect social media toxic comments by combining the feature extraction capabilities of Long Short Term Memoery (LSTM)-based neural networks with multiple machine learning models, including Random Forest, Logistic Regression, and K-Nearest Neighbors. High-dimensional feature representations from the neural network are integrated with predictions from traditional classifiers, and Random Forest optimizes the output weights to maximize performance. Evaluated on a Kaggle dataset, the proposed model achieves an accuracy of 89.78% and outperforms individual models in handling the complexity of toxic comments. However, challenges such as overfitting, computational overhead, and interpretability remain. Future work aims to address these limitations through improved data augmentation, explainability methods, and more scalable architectures.
BibTeX
@article{ma2022integrated,
  author = {Jiahuai Ma and Kaixian Xu and Yu Qiao and Zhaoyan Zhang},
  title = {{An Integrated Model for Social Media Toxic Comments Detection: Fusion of High-Dimensional Neural Network Representations and Multiple Traditional Machine Learning Algorithms}},
  journal = {Journal of Computational Methods in Engineering Applications},
  year = {2022},
  doi = {10.62836/jcmea.v2i1.0005},
}
Figure