TL;DR
In subjective tasks like cyberbullying severity, collapsing multiple annotator judgments into a single majority label discards vital information about human perception. This paper demonstrates that we can recover most of this lost "distributional signal" through simple post-hoc calibration of existing models, rather than expensive retraining, leading to more faithful and potentially fairer content moderation.
What Problem Does This Paper Address?
Machine learning for content moderation typically follows a "gold standard" paradigm: multiple humans label a comment, and their judgments are aggregated into a single majority vote. However, for subjective tasks like grading the severity of cyberbullying, disagreement isn't just noise—it's signal.
A comment that three people call "moderate" and two call "severe" is fundamentally different from one that all five agree is "moderate." Yet, standard training treats them identically. The authors find that in their primary dataset, a staggering 95.9% of comments had at least one dissenting annotator, and only 4.1% were unanimous. By forcing a consensus, we lose the "perspectivism" that reflects how different people perceive harm.

Methods
The researchers investigated whether models can predict the distribution of human judgments (e.g., 60% mild, 40% moderate) instead of just a single label. They built 11 different systems using a DeBERTa-v3-base backbone, varying architectures (flat vs. hierarchical) and training targets.
The core of their approach is Dirichlet calibration. Instead of retraining a model from scratch to output distributions, they took models trained on majority labels and applied a small, 12-parameter map to adjust the output probabilities after training. They compared these "calibrated twins" against models fully retrained on soft label distributions.
Datasets
The study utilized three primary datasets to test generalization:
- Insta-CTSR: An extension of the University of Colorado Instagram dataset containing ~106k comments. The study focused on 9,553 cyberbullying comments, each labeled by 5 annotators on a 4-level severity scale.
- EPIC: A binary irony dataset that includes annotator gender, used to test group fairness.
- ConvAbuse: A 5-level abuse-severity dataset used for replication.
Key Insights and Ideas
- The Accuracy Ceiling: Standard metrics like Macro-F1 "saturate" quickly. Two models might have nearly identical accuracy against the majority label but differ wildly in how well they represent the underlying human disagreement. Accuracy alone is a poor proxy for a model's competence in subjective domains.
- Calibration is a "Free Lunch": Most of the benefit of distribution-aware modeling comes from fixing overconfidence. Temperature scaling and Dirichlet calibration recovered 50-67% of the "closable gap" in fidelity to human labels without any retraining.
- The Shared Model Limit: While predicting distributions improves the fit for all demographic groups (e.g., both men and women), it doesn't close the performance gap between them. A single shared model can represent the average disagreement but cannot know which group is "right" for a specific comment without annotator metadata.
Key Results
- Disagreement is the Norm: Only 4.1% of cyberbullying comments received unanimous severity ratings.
- Fidelity Gains: Using Brier score (where lower is better) to measure fidelity to human distributions, the "raw" majority-trained model scored 0.294. Simple temperature scaling dropped this to 0.182, and the full label-distribution map reached 0.144. Full retraining only added a tiny further gain to 0.137.
- Fairness: On the EPIC irony dataset, preserving the distribution improved the Brier score for men's labels by 0.007 and women's by 0.005 compared to the calibrated majority baseline.
Limitations and Future Work
The authors acknowledge several boundaries to their evidence:
- Era and Language: The data is from Instagram (2013-2015) and is entirely in English.
- Annotator Identity: The primary dataset lacks annotator demographics, forcing the use of the EPIC dataset for fairness analysis.
- Human-in-the-loop: While the model represents disagreement better, it’s still an open question whether showing these distributions to human moderators actually helps them make better or faster decisions.
- Severe Class Performance: Interestingly, the benefits of distribution training were mostly seen in the "mild" and "moderate" classes; for the rarest "severe" cases, the majority-trained model remained competitive.
Future work suggests prompting or tuning "frontier" LLMs with label distributions to see if they can better capture the nuances of human subjectivity.
Summary updated