GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectives

Vinodkumar Prabhakaran, Christopher Homan, Lora Aroyo, Aida Mostafazadeh Davani, Alicia Parrish, Alex S. Taylor, Mark Diaz, Ding Wang, Gregory Serapio-García

TL;DR

This paper introduces GRASP, a novel framework for analyzing systematic disagreements in human annotations across different demographic groups. By quantifying in-group and cross-group cohesion, GRASP helps identify which groups hold significantly different perspectives on subjective tasks, offering a more nuanced understanding of annotation diversity and its implications for AI development.

What Problem Does This Paper Address?

Human annotation is crucial for training and evaluating AI models, especially for tasks involving subjective judgments like safety and offensiveness. However, existing methods often overlook the inherent subjectivity of human raters, leading to a "one-size-fits-all" approach that can marginalize minority perspectives and amplify societal biases. While previous work has hinted at demographic differences in annotations, there's a lack of generalizable methods to systematically measure and understand these group-level associations in perspectives. This paper aims to fill that gap by providing a framework to quantify and statistically validate these differences.

Methods

The core of the GRASP framework lies in two key concepts:

  • In-group Cohesion ($C_I$): Measures how much raters within a specific demographic group agree with each other.
  • Cross-group Cohesion ($C_X$): Measures how much one rater group agrees with another group.

These cohesion metrics are combined into a Group Association Index (GAI), which quantifies the strength of a demographic group's association with a particular annotation pattern. A GAI greater than 1 suggests higher agreement within the group compared to outside, indicating a distinct perspective.

To ensure the reliability of these measurements, GRASP employs permutation tests for statistical significance testing. This approach accounts for the dependencies within the data, which is crucial for annotation datasets. The framework is designed to be flexible, allowing for various underlying metrics for cohesion and divergence, and can analyze groups defined by any combination of demographic attributes.

Datasets

The GRASP framework was applied to two existing datasets:

  • DICES-350: A dataset of 350 chatbot conversations annotated for safety by 104 raters. This dataset was specifically curated to have a diverse pool of raters across age, gender, and race.
  • D3: A dataset of social media posts annotated for offensiveness by 4000 raters, balanced across cultural regions, gender, and age.

Key Insights and Ideas

  • Subjectivity is Systematic: Disagreements in human annotations aren't just random noise; they can be systematic and tied to demographic factors.
  • Beyond Majority Vote: Relying solely on majority votes can obscure important minority perspectives and reinforce existing biases. GRASP offers a way to quantify these differences.
  • Demographics Matter, But How?: The framework helps identify which demographic axes (e.g., race, gender, age, or intersections thereof) are most influential for specific tasks.
  • Task-Dependent Associations: The influence of demographic factors on annotation perspectives varies significantly depending on the task (e.g., safety vs. offensiveness).
  • Intersectional Analysis is Key: Analyzing demographic groups in isolation is insufficient; understanding intersections (e.g., Latine women) reveals more nuanced patterns of agreement and disagreement.

Key Results

The application of GRASP to DICES-350 and D3 revealed several key findings:

  • DICES-350 (Safety Annotations):
  • Race/ethnicity showed significant results on its own, suggesting age and gender were less influential individually.
  • Intersectional analysis highlighted Latine women as having the highest in-group cohesion (0.238).
  • White men had the highest GAI (1.262), indicating a strong association with a particular perspective, while Asian women had the lowest GAI (0.540), suggesting more agreement with raters outside their group.
  • D3 (Offensiveness Annotations):
  • Age and region groups showed significant associations.
  • 50+-year-old raters from Sinosphere had the highest GAI (2.225), followed by non-binary genders (2.172).
  • Intersectional results showed that while women generally didn't report high GAI, specific subgroups of women in different regions exhibited more in-group agreement.
  • The study also found that different metrics (IRR, Negentropy, Plurality size) capture different aspects of agreement and can yield varying results, emphasizing the need to choose metrics carefully based on the task.

Limitations and Future Work

  • Demographic Simplification: The demographic breakdowns in the datasets are simplified representations of the population. More granular data could reveal further insights.
  • Other Intersectional Groups: The paper focused on specific intersectional groups; further research could explore other combinations.
  • Distinguishing "Good" vs. "Bad" Disagreement: While GRASP identifies systematic disagreements, further work is needed to disentangle disagreements that represent valuable diverse perspectives from those that might indicate issues like lack of training or rater bias.
  • Metric Selection: The paper notes that the choice of metric can influence results, and future work should investigate which metrics are most suitable for different tasks and data characteristics.
  • Dynamic Data Collection: The framework could be extended to enable dynamic data collection that adapts to emergent group associations.
  • Understanding Reasons for Disagreement: Further investigation is needed to understand the underlying reasons (e.g., individual moral values) behind the observed systematic disagreements.

Summary updated

Abstract

Human annotation plays a core role in machine learning -- annotations for supervised models, safety guardrails for generative models, and human feedback for reinforcement learning, to cite a few avenues. However, the fact that many of these human annotations are inherently subjective is often overlooked. Recent work has demonstrated that ignoring rater subjectivity (typically resulting in rater disagreement) is problematic within specific tasks and for specific subgroups. Generalizable methods to harness rater disagreement and thus understand the socio-cultural leanings of subjective tasks remain elusive. In this paper, we propose GRASP, a comprehensive disagreement analysis framework to measure group association in perspectives among different rater sub-groups, and demonstrate its utility in assessing the extent of systematic disagreements in two datasets: (1) safety annotations of human-chatbot conversations, and (2) offensiveness annotations of social media posts, both annotated by diverse rater pools across different socio-demographic axes. Our framework (based on disagreement metrics) reveals specific rater groups that have significantly different perspectives than others on certain tasks, and helps identify demographic axes that are crucial to consider in specific task contexts.
BibTeX
@inproceedings{prabhakaran2024grasp,
  author = {Vinodkumar Prabhakaran and Christopher Homan and Lora Aroyo and Aida Mostafazadeh Davani and Alicia Parrish and Alex S. Taylor and Mark Diaz and Ding Wang and Gregory Serapio-García},
  title = {{GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectives}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.190},
}
Figure