Introduction
As the world’s third-largest democracy, Indonesia navigates a digital landscape where political fervor frequently transitions into online hostility. While the link between political polarization and toxicity is often observed qualitatively, quantifying this relationship—and understanding how individual backgrounds influence the perception of "toxic" content—remains a significant challenge for Natural Language Processing (NLP).
In a recent study, Susanto et al. introduced IndoToxic2024, the first multi-labeled Indonesian dataset designed to capture the intersection of toxicity, polarization, and annotator demographics. Their research suggests that effective content moderation requires a shift from isolated analysis to a holistic approach that considers the identity of the person performing the labeling.
The IndoToxic2024 Dataset
The primary contribution of this work is the IndoToxic2024 dataset, which comprises 28,477 unique Indonesian social media texts sourced from X (Twitter), Facebook, Instagram, and various news platforms. Unlike traditional datasets, IndoToxic2024 is multi-labeled, covering:
- Toxicity: Including five subtypes: Insults, Threats, Profanity, Identity Attacks, and Sexually Explicit content.
- Polarization: Identifying divisive "us vs. them" rhetoric.
- Annotator Demographics: Detailed profiles of the 29 annotators, including ethnicity, religion, age, education, domicile, disability status, and 2024 presidential leaning.
To leverage this demographic data, the researchers utilized an "exploded" dataset format. In this structure, each text is linked to a specific coder’s profile, allowing models to treat annotator background as an input feature rather than just a source of ground truth.
Methodology and Benchmarking
The study evaluated the dataset using a variety of models to establish performance baselines. The researchers compared specialized Indonesian models, such as IndoBERTweet and NusaBERT, against prominent Large Language Models (LLMs) including GPT-4o and Llama 3.1.
A key methodological innovation involved cross-task feature engineering. The team tested whether a text's "polarization score" could improve the prediction of its "toxicity" and vice versa, exploring the symbiotic relationship between these two linguistic phenomena.
Key Insights
1. The Subjectivity of Polarization
The study found that polarization is significantly more subjective than toxicity. The inter-coder agreement (measured via Gwet’s AC1) for polarization ranged from 0.37 to 0.39, compared to 0.61 for toxicity. This indicates that what one individual perceives as "polarizing" is heavily dependent on their personal identity and political leanings.
2. The Toxicity-Polarization Nexus
The data revealed a strong correlation between toxic language and divisive rhetoric. Interestingly, while 48% of toxic texts during the 2024 election cycle were also polarizing, only 25% of polarizing texts were classified as toxic. This suggests that discourse can be deeply divisive without necessarily employing profanity or overt harassment.
3. Identity as a Predictive Feature
Incorporating demographic information—specifically ethnicity, domicile, and religion—significantly improved polarization classification. By including these features, models were better able to "understand" specific demographic viewpoints, leading to more accurate predictions of how different groups might perceive a statement.
4. LLMs and the Nuance Gap
Despite their general capabilities, LLMs like GPT-4o-mini struggled to utilize demographic information effectively. These models often underperformed compared to smaller, domain-specific BERT models. The researchers hypothesize that LLMs may prioritize their internal safety training over the specific demographic "personas" provided in prompts.
Results and Performance
IndoBERTweet emerged as the most effective model, achieving an accuracy of 0.844 for toxicity and 0.801 for polarization. By combining featural information (using polarization labels to assist in toxicity detection) with demographic data, the researchers achieved an F1@1 score of 0.765 for toxicity—a notable improvement over models using text alone.
Conclusion and Future Directions
The IndoToxic2024 dataset represents a move toward "identity-aware" AI. However, the authors acknowledge several limitations, including the inherent noise in polarization labels due to subjectivity and the relatively small size of the annotator pool (29 individuals) relative to Indonesia's population of 277 million.
Future research will focus on optimizing LLMs to better leverage demographic context and expanding the annotator pool to capture an even broader spectrum of the Indonesian social fabric. By understanding the "who" behind the label, NLP systems can become more nuanced tools for fostering healthy digital discourse.
Summary updated