Abstract
Hate speech detection (HSD) is critical for maintaining healthy online discourse but is often hampered by context scarcity in short text snippets and the limited availability of paired multimodal datasets. We present TrusV-HSD, a novel framework that addresses these challenges by generating "imaginary" visual cues from text using diffusion models. Our approach combines these synthetic images with a lightweight Mamba-based detector and a Dirichlet-based "trustworthy loss" to quantify prediction uncertainty. Experimental results across three benchmarks demonstrate that TrusV-HSD achieves state-of-the-art performance, particularly in handling implicit hate and short-form content.
Introduction
Social media platforms face significant hurdles in automating hate speech detection. First, hate speech is frequently conveyed through short, punchy text that lacks sufficient semantic depth for confident classification. Second, while multimodal models (text + image) generally outperform text-only models, most real-world hate speech is not accompanied by images. Finally, the inherent subjectivity of hate speech leads to high uncertainty and potential bias in model predictions.
TrusV-HSD bridges this gap by "hallucinating" visual context. By generating images that capture the metaphors and concrete concepts within text, the model gains a multidimensional representation of the input, allowing it to "see" the intent behind the words.
Methodology
The TrusV-HSD framework integrates three core components designed for efficient multimodal fusion and reliability:
- Visual Cues Generation Module: Utilizing a pre-trained image generator (e.g., Stable Diffusion), the system synthesizes visual imagery based on input text. This provides additional context without requiring existing paired data.
- Mamba-based Detector: To fuse text features (from fBERT) and image features (from CLIP), we employ a Connection Block followed by a Mamba Block. Mamba, a State Space Model (SSM), offers linear computational complexity, making it more efficient than Transformers for long-range multimodal dependencies.
- Trustworthy Loss: We replace standard cross-entropy loss with a Dirichlet-based loss inspired by Evidential Deep Learning. This allows the model to treat predictions as a distribution, effectively quantifying uncertainty and down-weighting unreliable or biased samples during training.
Experimental Results
The model was evaluated on the SemEval2019 Task-5 (SE), Fox News User Comments (FNUC), and Implicit Hate Speech (IHC) datasets. TrusV-HSD demonstrated significant improvements over both text-only and existing multimodal baselines:
- Explicit Hate: On the SE dataset, it achieved an F1 score of 78.92%, compared to 71.37% for the fBERT baseline.
- Implicit Hate: On the IHC dataset, it reached an F1 of 71.19%, outperforming specialized models like RSGNN.
- Robustness to Speech Length: A key finding was the model's ability to maintain high accuracy across varying text lengths. While standard models often overfit the distribution of short texts (which comprise 86% of the SE dataset), TrusV-HSD uses Trustworthy Loss to manage the uncertainty inherent in longer, more subjective posts.
Conclusion and Future Work
TrusV-HSD offers a robust, "backbone-free" architecture for hate speech detection that leverages synthetic visual augmentation. While the computational cost of image generation remains a challenge for real-time applications, the framework's ability to quantify uncertainty marks a significant step toward more reliable social media moderation. Future research will explore multilingual applications and the integration of user-specific features to further reduce subjectivity.
Summary updated