Dissecting Biases in Relation Extraction: A Cross-Dataset Analysis on People’s Gender and Origin

Marco Stranisci, Pere-Lluís Huguet Cabot, Elisa Bassignana, Roberto NavigliACL 2024

TL;DR

This paper introduces a novel, transparent methodology to dissect gender and origin biases in Relation Extraction (RE) systems, analyzing both training data and model predictions across multiple datasets. Their findings reveal that biases are a pervasive, compounded issue, affecting underrepresented groups in both data and model outputs, and that simple debiasing strategies can introduce new problems.

What Problem Does This Paper Address?

Relation Extraction (RE) is a crucial component of Natural Language Understanding (NLU) systems, enabling tasks like knowledge base population and question answering. However, like many NLP technologies, RE systems are susceptible to biases. While previous work has touched upon gender bias in RE, this paper highlights a critical gap: the need for a comprehensive, transparent analysis that goes beyond gender to include other dimensions like origin, and that examines biases at multiple stages of the RE pipeline, from data curation to model predictions. The authors argue that existing methods often overlook allocative biases (unequal distribution of opportunities) in favor of representational biases (stereotypical associations), and that a more nuanced, multi-axial approach is needed.

Methods

The authors propose a four-step methodology for bias analysis in RE:

  1. Label Alignment: They create a unified taxonomy of relation types to enable cross-dataset comparability, mapping the diverse label sets of different RE datasets into a common space. This is crucial for comparing results across corpora.
  2. Relation Extraction: They extract semantic triplets (subject, object, relation) involving "person" entities from RE resources. This can be done using existing RE datasets or model predictions.
  3. Socio-Demographic Information Collection: Using Entity Linking (EL) techniques, they gather meta-information such as "gender" and "place of birth" for the identified "person" entities, primarily from Wikidata.
  4. Bias Analysis: Finally, they analyze the distribution of extracted triplets across different social groups (e.g., men vs. women, Global North vs. Global South) to identify potential allocative and representational imbalances. This involves examining both the training data and the predictions of RE models.

Datasets

The paper utilizes three widely adopted RE datasets for their analysis:

  • SREDFM: A distantly annotated dataset built on Wikipedia pages and Wikidata, employing a novel triplet critic filtering.
  • CrossRE: A multi-domain, manually annotated dataset covering news, politics, science, music, literature, and AI.
  • NYT: A dataset of news sentences from the New York Times corpus, distantly annotated using FreeBase.

They also use Wikidata as a source for socio-demographic information and ReLiK (a Retriever-Reader model) as their state-of-the-art RE approach for prediction analysis.

Key Insights and Ideas

  • Bias is a Compounded Issue: The study reveals that biases in RE are not isolated but rather a complex interplay of factors affecting underrepresented groups across data and model predictions.
  • Beyond Gender: The paper successfully extends bias analysis to include "place of birth" (Global North vs. Global South), broadening the scope of understanding RE biases.
  • Transparency is Key: The proposed methodology emphasizes interpretability, allowing researchers to pinpoint where biases manifest in the RE pipeline.
  • Simple Debiasing Can Backfire: Balancing data for one underrepresented group (e.g., gender) can inadvertently introduce or amplify biases for other groups (e.g., origin).
  • Taxonomy is Crucial for Cross-Dataset Analysis: The developed relation type taxonomy is essential for making RE datasets comparable and enabling robust bias analysis across different resources.

Key Results

  • Underrepresentation: Women and people from the Global South are significantly underrepresented in the analyzed RE corpora, often falling between 10-20% of the total, despite the Global South comprising ~80% of the world's population.
  • Allocative Bias: The analysis of predictions shows that models can exhibit allocative harms, with varying effects depending on the dataset and whether pre-training is used. For instance, pre-training on SREDFM amplified allocative bias for women in CrossRE but mitigated it for people from the Global South.
  • Representational Bias: Similar associations between groups and relation types were observed in both training data and predictions. For example, "relationship" is often associated with women, and "geographical" with people from the Global South. The NYT corpus showed a higher number of biases, with men and Northern people appearing in relations emphasizing profession.
  • Model Performance vs. Bias: While transfer learning (using SREDFM for pre-training) improved RE performance on smaller datasets like CrossRE, it had a mixed effect on bias mitigation, amplifying some biases while reducing others.

Limitations and Future Work

  • Taxonomy Granularity: The proposed relation type taxonomy, while enabling cross-dataset analysis, might be too broad in some categories, potentially obscuring more specific biases.
  • Wikidata Limitations: The reliance on Wikidata for socio-demographic information is a limitation, as Wikidata itself exhibits underrepresentation of women and Global South populations, potentially skewing the analysis. The paper acknowledges that identified biases might reflect Wikidata's limitations as much as the RE systems'.
  • Binary Gender Model: The study is constrained by Wikidata's binary representation of gender (due to property P21), preventing the analysis of non-binary identities. Future work could explore knowledge bases with richer representations of gender.
  • Intersectional Bias: The authors suggest that an intersectional approach, jointly considering multiple dimensions of underrepresentation (like gender and origin), is needed for a more comprehensive understanding and mitigation of biases.
  • Bias Origin: The paper focuses on how biases are reflected in RE resources rather than pinpointing their exact origin, acknowledging that biases can emerge at various stages of the NLP pipeline.

Summary updated

BibTeX
@inproceedings{stranisci2024dissecting,
  author = {Marco Stranisci and Pere-Lluís Huguet Cabot and Elisa Bassignana and Roberto Navigli},
  title = {{Dissecting Biases in Relation Extraction: A Cross-Dataset Analysis on People’s Gender and Origin}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure