Toward Deconfounding the Effect of Entity Demographics for Question Answering Accuracy

Maharshi Gor, Kellie Webster, Jordan L. Boyd-Graber

Abstract

Popular Question Answering (QA) datasets are heavily skewed toward Western men, raising concerns about algorithmic bias. This research investigates whether this representational skew leads to performance gaps for underrepresented demographic groups. By auditing four major benchmarks—Natural Questions (NQ), Quizbowl, SQuAD, and TriviaQA—and using logistic regression for deconfounding, the authors find that performance variations are primarily driven by the topic of the question rather than the entity's identity.

Introduction

In the field of Natural Language Processing (NLP), datasets often mirror societal biases. Most English-language QA datasets are notoriously "WEIRD" (Western, Educated, Industrialized, Rich, and Democratic), focusing disproportionately on men from the US and UK. This study addresses a critical fairness question: does this representational skew result in a performance gap? For instance, if a model is trained primarily on American male politicians, does it underperform when asked about female scientists from the Global South? This is the "deconfounding" challenge—separating the influence of a person's identity (gender, nationality) from the difficulty of the subject matter (profession).

Methodology

The researchers developed a multi-stage pipeline to audit four major QA benchmarks: Natural Questions (NQ), Quizbowl (QB), SQuAD, and TriviaQA. The approach involved:

  1. Entity Linking: Identifying "people" entities within questions, answers, and source documents using Google Cloud-NL.
  2. Demographic Mapping: Linking those entities to Wikidata to extract three key characteristics: gender, nationality, and professional field.
  3. Statistical Auditing: Using $\chi^2$ (Chi-squared) tests to see if accuracy was independent of demographics.
  4. Logistic Regression for Deconfounding: To ensure they weren't just seeing correlations, they built a logistic regression model to predict question correctness. They included "difficulty features" (like how often an entity appeared in training or the linguistic similarity between the question and evidence) alongside demographic features to see which truly influenced the outcome.

Representational Skew and Performance

The study confirmed a massive representational gap across all datasets: men accounted for 75% to 95% of entities, and US/UK individuals dominated the nationality charts. However, the performance results presented a paradox.

Model accuracy across gender, nationality, and profession for NQ, Quizbowl, SQuAD, and TriviaQA.

As shown in the figure above, accuracy for women was often comparable to or even higher than for men. In the Natural Questions (NQ) dataset, for example, women had an 82.14% accuracy rate compared to 70.98% for men. Similarly, no significant relationship was found between a person's national origin and the model's ability to answer a question about them across any of the four datasets.

The Deconfounding Effect: Topic is King

The core finding is that "topic" or profession is the primary driver of accuracy. In NQ, questions about entertainers (where women have higher representation) are generally easier for models than questions about scientists. This creates a "Simpson’s Paradox" effect where women appear to have higher accuracy simply because they are more frequently associated with "easier" categories in the dataset.

Once the researchers controlled for profession and question type using logistic regression, gender was no longer a statistically significant predictor of accuracy. Profession, however, remained highly influential. In Quizbowl, Science/Tech questions were significantly harder (negative coefficient in the regression model), while Writing and History were more predictable.

The "Not Found" Penalty

A consistent demographic "bias" was observed for entities that did not have a Wikidata entry. Models performed significantly worse on these "unknown" individuals. This highlights that the lack of structured knowledge in external databases is a more significant hurdle for QA models than the demographic category of the entity itself.

Conclusion and Future Work

While the models studied do not show inherent performance bias against women or non-Westerners in these datasets, the authors warn that the extreme data skew limits the ability to detect real biases against specific intersectional groups (e.g., female scientists from India).

Future work should move toward counterfactual evaluation, systematically swapping a male name for a female name in the same question to observe changes in model behavior. Ultimately, while current performance might not be "biased," the datasets fail to represent global diversity, which limits our ability to verify if these systems are fair for everyone.

Summary updated

BibTeX
@inproceedings{gor2021toward,
  author = {Maharshi Gor and Kellie Webster and Jordan L. Boyd-Graber},
  title = {{Toward Deconfounding the Effect of Entity Demographics for Question Answering Accuracy}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.444},
}
Figure