A Comprehensive Study of Gender Bias in Chemical Named Entity Recognition Models

Xingmeng Zhao, Ali Niazi, Anthony RiosNAACL 2024 · DOI

TL;DR

This paper investigates gender bias in chemical Named Entity Recognition (NER) models, finding that these systems disproportionately misclassify female-associated names as chemicals and perform worse on data related to women. The authors introduce a new annotated dataset and a framework to quantify these biases, urging practitioners to address them in downstream applications.

What Problem Does This Paper Address?

Chemical Named Entity Recognition (NER) is a crucial step in many biomedical and pharmaceutical applications, from identifying adverse drug reactions to facilitating drug discovery. However, it's unclear if these powerful AI models perform equally well for everyone. Performance disparities can lead to unequal benefits or even harm. This paper specifically tackles the problem of gender bias in chemical NER systems, exploring whether these models exhibit different performance characteristics when processing text associated with different genders. This is particularly important because chemical NER is often a foundational component in systems that can directly impact health outcomes.

Methods

The researchers employed a two-pronged approach to investigate gender bias:

  • Synthetic Data Generation: They created a synthetic dataset using templates like "[NAME] said they have been taking [CHEMICAL] for an illness." By systematically filling in male and female names (based on popular baby names from different decades) and known chemicals, they could isolate and quantify specific types of errors, such as whether female names are misclassified as chemicals. This method allows for controlled experiments to understand the impact of morphological patterns and naming conventions.
  • Real-World Data Analysis: They curated a novel dataset from the Reddit community r/AskDocs, which contains posts from individuals seeking medical advice. This dataset was annotated for chemical entities and included self-identified gender mentions (e.g., "I [25F]"). This real-world data allows for the evaluation of model performance on naturally occurring text, reflecting how people actually write and discuss health-related topics.

For model evaluation, they used state-of-the-art NER architectures, specifically Bi-LSTM+CRF, and experimented with various embedding types: Word2Vec, Flair (contextual character-level embeddings), and BERT/BioBERT (transformer-based subword embeddings). Bias was measured by comparing precision, recall, and F1 scores between male and female data splits.

Datasets

The study utilized a combination of existing and newly created datasets:

  • Publicly-Released Datasets:
    • CDR: A corpus for chemical-disease relation extraction, used here for chemical NER.
  • CHEMDNER: A corpus specifically designed for chemical entity recognition from biomedical literature.
  • CHEBI: A comprehensive dataset containing annotated chemicals, proteins, and biological activities.
  • Newly Curated Datasets:
    • Synthetic (Template) Data: Generated by the authors using predefined templates and lists of names and chemicals to systematically study bias.
  • AskDocs Dataset: A novel corpus collected from the r/AskDocs subreddit, featuring self-identified gender mentions and annotated chemical entities. This is highlighted as the first Reddit-based Chemical NER dataset with self-identified gender information.

Key Insights and Ideas

  • Female Names as Chemicals: A striking finding is that female names are disproportionately misclassified as chemicals, especially when they appear as brand names. This suggests that naming conventions, which sometimes use feminine-sounding suffixes or patterns, can directly influence model behavior.
  • Performance Gap in Real-World Data: The study confirmed that chemical NER models exhibit performance disparities on real-world data associated with different genders. Models generally performed better on data linked to males.
  • Contraceptives and Gender Bias: The models showed particularly poor performance in identifying chemicals frequently associated with women, such as contraceptives. This highlights a critical area where bias could lead to significant downstream issues in health applications.
  • Embedding Type Matters: The choice of word embeddings can influence the degree of bias. While all models showed some bias, contextual embeddings like Flair and BERT/BioBERT sometimes exhibited more pronounced biases compared to traditional word embeddings, though the relationship is complex and depends on the training data.

Key Results

  • Synthetic Data: The synthetic data analysis revealed that models generally perform differently on male versus female templates, with PubMed-trained embeddings showing an average precision bias of .0242 against female names, and general embeddings showing a larger bias of .0407.
  • AskDocs Data: On the AskDocs dataset, models trained on existing corpora (CDR, CHEMDNER) and tested on male/female splits showed significant differences. For instance, the ratio of false negatives for contraceptives was notably higher for the female group.
  • False Negative Rates: Table 7 shows that for categories like "Contraceptives," the false negative rate (FNR) for females was 0.9730, compared to 1.0000 for males, indicating a slight but present bias. However, the weighted FNR (wFNR) metric, which accounts for category frequency, revealed a much larger gap (0.1213 vs. 0.0116), suggesting that many false negatives are concentrated in gender-specific categories for the female group.

Limitations and Future Work

  • Binary Gender Assumption: The study acknowledges that it focuses on binary gender representations and does not account for non-binary individuals. Additionally, the use of popular baby names might not adequately represent immigrant populations, potentially making the results European-specific.
  • Data Annotation: The adjudication of disagreeing items in the AskDocs dataset relied on a single graduate student, which could introduce human error. The sheer volume of data also made a comprehensive manual review challenging.
  • Future Directions: The authors suggest exploring biases in prompting-based NER solutions with large language models (LLMs) and investigating how chemical NER biases impact downstream tasks like relationship classification and question answering. They also plan to explore data collection methods beyond binary gender to include a wider range of individuals.

Summary updated

BibTeX
@inproceedings{zhao2024comprehensive,
  author = {Xingmeng Zhao and Ali Niazi and Anthony Rios},
  title = {{A Comprehensive Study of Gender Bias in Chemical Named Entity Recognition Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.245},
}
Figure