TL;DR
This paper introduces PerspectiveMod, a novel dataset and framework for understanding the subjective nature of online discussion moderation. By capturing detailed annotator perspectives, they reveal that moderation decisions are deeply influenced by individual experiences, attitudes, and values, not just objective comment properties, paving the way for more nuanced AI moderation tools.
What Problem Does This Paper Address?
Online platforms fostering deliberative and argumentative discourse, like civic participation portals and forums, rely heavily on effective moderation. However, current NLP approaches to moderation primarily focus on detecting toxic or harmful content, falling short in more complex deliberative contexts. Here, moderation involves nuanced judgments about comment quality, clarity, and contribution to the discussion, which are inherently subjective and context-dependent. Existing datasets often assume a single "ground truth," failing to capture the diversity of interpretations and the factors that shape them. This gap makes it challenging to build AI systems that can truly assist human moderators in these complex environments.
Methods
The authors introduce PerspectiveMod, a dataset built around a "perspectivist" approach. This means they don't assume a single correct moderation decision but instead document the variety of interpretations. They collected annotations from both expert moderators and trained non-expert crowdworkers. Crucially, they went beyond simple "moderate/don't moderate" labels by collecting detailed information on:
- Moderation necessity and priority: When and how urgently a comment needs attention.
- Moderation functions: The specific actions a moderator might take (e.g., flagging, clarifying, broadening discussion).
- Comment-level discourse properties: Fine-grained aspects of the comment related to argumentation and deliberative theory (e.g., clarity, constructiveness, tone).
- Annotator-level metadata: This is a key innovation. They collected detailed information through structured interviews about annotators' backgrounds, experience with online moderation, attitudes towards AI, and their personal values regarding online discussions.
They used statistical modeling (generalized linear mixed models) to analyze how these annotator-specific factors and comment properties influence moderation decisions.
Datasets
The primary contribution is the introduction of the PerspectiveMod dataset. This dataset consists of 225 comments, each annotated 12 times, totaling over 1,645 annotation hours. The comments were sampled from three distinct moderation ecosystems:
- RegulationRoom: A civic engagement platform with professional moderators.
- r/ChangeMyView (CMV): A subreddit with semi-trained community moderators.
- UMOD: A dataset derived from CMV, focusing on informal, user-driven moderation.
This multi-source approach captures a spectrum of moderation practices, from expert-led to grassroots.
Key Insights and Ideas
- Moderation is deeply subjective: The study strongly confirms that moderation decisions are not solely based on objective comment features. Annotator background, experience, and personal values play a significant role.
- Expert vs. Crowdworker Differences: Experts tended to be more conservative in moderating, while crowdworkers showed greater variability. Experts also focused on different quality dimensions compared to crowdworkers.
- "Negative" vs. "Positive" Moderation: Crowdworkers were more influenced by "negative" cues (misinformation, disrespect, inappropriateness), while experts leaned towards "positive" moderation, focusing on constructiveness and enriching the discussion.
- AI's Role is Complex: Annotators had diverse views on AI's role, ranging from simple flagging to complex task support. Those who saw AI as a tool for protecting users from harm were more sensitive to emotion and misinformation, while those who viewed AI as a complex support tool were less reactive to individual cues.
- Moderation Behavior is Controversial: Even "moderation-like behavior" (users pointing out issues) was interpreted differently, with some seeing it as constructive and others as policing.
Key Results
- Inter-annotator agreement was low: This underscores the inherent subjectivity of the task, even among experts. Fleiss' kappa for experts was 0.055.
- Crowdworkers explained variance: Comment properties explained 38% of the variance in crowdworker moderation decisions, with misinformation, respect, and appropriateness being strong predictors.
- Experts' decisions are more idiosyncratic: For experts, comment properties explained only 13% of the variance, with the remaining 30% attributed to annotator-specific differences. Constructiveness was the strongest predictor for experts.
- Annotator characteristics matter: Active participation in discussions, annotator confidence, and perceived task difficulty all influenced moderation decisions. For example, active participants were more sensitive to appropriateness.
Limitations and Future Work
The authors acknowledge several limitations:
- Dataset Size: The dataset is relatively small, which may constrain its use for training large-scale models. Future work could expand the data across more platforms.
- Unobserved Factors: Factors like mood, platform familiarity, or implicit biases were not captured but likely influence decisions.
- Limited Diversity: The annotator pool lacked geographic and cultural diversity, which could affect interpretations of norms like tone or argumentative style.
Future work should focus on:
- Modeling Moderation: Developing models that can meaningfully engage with the perspectivist nature of moderation, serving as decision aids rather than replacements for human judgment.
- Expanding Data: Collecting data from a wider range of platforms and discourse settings.
- Capturing More Nuance: Developing methods to capture a fuller range of influences on human judgment.
Summary updated