Voice for the Voiceless: Active Sampling to Detect Comments Supporting the Rohingyas

Shriphani Palakodety, Ashiqur R. KhudaBukhsh, Jaime G. CarbonellAAAI 2020 · DOI

Introduction

Social media moderation typically focuses on the removal of harmful content, such as hate speech. However, for persecuted minorities like the Rohingya refugees, the challenge is not just the presence of vitriol, but the absence of supportive voices. This paper introduces the concept of "help speech"—comments that defend or support marginalized groups—and proposes a method to actively surface these rare positive sentiments from massive online datasets.

The Challenge of Detecting Help Speech

Persecuted groups often face a digital void. Language barriers, limited internet access, and overwhelming negative sentiment from global audiences mean that supportive comments are often "needles in a haystack." In a corpus of hundreds of thousands of YouTube comments, the authors found that supportive voices are extremely rare. Standard random sampling fails to build effective machine learning models because it captures too few positive examples to define the class accurately.

Methodology: An Active Learning Pipeline

To address the extreme class imbalance, the researchers developed a five-stage Active Learning framework. This pipeline iteratively refines a classifier by strategically selecting which comments to label.

The multi-stage active learning pipeline used to build the help speech dataset.

The process begins with a small Seed Set and Random Sampling to establish a baseline. The core innovation lies in Nearest-Neighbor (NN) Sampling, which uses Sent2Vec document embeddings to find unlabeled comments semantically similar to known supportive ones. This is followed by Certainty Sampling to balance the dataset with high-confidence positives and Uncertainty Sampling to clarify the model's decision boundary. The final model is a Support Vector Machine (SVM) utilizing text n-grams and document embeddings.

Dataset and Discourse Analysis

The study utilized a massive dataset of 263,482 YouTube comments from over 113,000 users across 5,153 videos related to the Rohingya crisis. These videos spanned 19 countries, providing a global perspective on the conflict.

Word clouds comparing descriptors for "Rohingyas are" vs. "Rohingyas are not," highlighting the prevalence of negative sentiment.

Analysis of the discourse revealed a stark asymmetry. As shown in the word clouds, descriptors like "illegal," "terrorist," and "bengali" dominate the conversation. This negative framing underscores the difficulty of finding "help speech," which the authors define through criteria such as advocating for rights, condemning atrocities, and expressing sympathy.

Key Results

The active learning approach proved highly effective:

  • Efficiency: NN sampling was three times more effective at identifying supportive comments than random sampling.
  • Accuracy: The final SVM model achieved an F1-score of 78.28%.
  • Real-world Impact: In a test of the model's top 100 predictions, 88% were true positives, compared to a mere 10.6% hit rate from random sampling.

Conclusion

By shifting the focus from deletion to amplification, this research provides a tool for human rights organizations and moderators to find and support the voices of the marginalized. While challenges remain—such as distinguishing "help speech" from aggressive counter-speech—the framework is adaptable to other humanitarian crises, offering a path toward a more balanced digital discourse.

Summary updated

BibTeX
@inproceedings{palakodety2020voice,
  author = {Shriphani Palakodety and Ashiqur R. KhudaBukhsh and Jaime G. Carbonell},
  title = {{Voice for the Voiceless: Active Sampling to Detect Comments Supporting the Rohingyas}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i01.5382},
}
Figure