Assessing Group-level Gender Bias in Professional Evaluations: The Case of Medical Student End-of-Shift Feedback

Emmy Liu, Michael Henry Tessler, Nicole Dubosh, Katherine Hiller, Roger Levy

TL;DR

This paper investigates gender bias in medical student evaluations by analyzing narrative feedback using a fine-tuned BERT model. While finding no evidence of systematic bias in the overall evaluation scores, the study reveals that feedback for female students disproportionately mentions family and children.

What Problem Does This Paper Address?

Despite women making up roughly 50% of medical school graduates, they remain underrepresented in senior positions and earn less than their male counterparts. This disparity is often linked to subtle forms of gender bias that can manifest early in a physician's career, potentially snowballing into significant career disadvantages. Previous research on gender bias in evaluations, particularly in academia and medicine, has often relied on broad linguistic analysis (like LIWC) or focused on specific documents like recommendation letters. This paper aims to address the gap by examining day-to-day feedback given to medical students during their clinical shifts, using a more sophisticated natural language processing approach to detect potential group-level gender bias in a more nuanced way.

Methods

The core of this study employs a fine-tuned BERT (Bidirectional Encoder Representations from Transformers) model. The researchers fine-tuned a pre-trained BERT model on a dataset of medical student evaluations, specifically training it to predict the global competency score from the free-text comments. The key innovation lies in analyzing the residuals of these predictions. By comparing the model's predicted score to the actual score given by the human assessor, they can identify instances where the text might suggest a different score than what was assigned. If there's a systematic difference in these residuals between male and female students, it suggests that assessors might be writing about students differently based on gender, even when their objective performance is similar. This approach moves beyond simple keyword analysis by capturing more complex linguistic patterns. They also conducted a traditional LIWC (Linguistic Inquiry and Word Count) analysis for comparison.

Datasets

The study utilizes a dataset of written and quantitative assessments of medical student performance collected through the National Clinical Assessment Tool (NCAT-EM). This tool is a standardized assessment based on direct observation of clinical skills, implemented across 13 institutions in the United States from 2017-2019. The dataset includes free-text comments, categorical skill assessments, a global competency score (on a 0-3 scale), and demographic information about students and assessors (gender, age, rank). After preprocessing (removing names and gendered pronouns), the dataset comprised 3162 individual assessments, with 1767 for male students and 1395 for female students. The data was split into training (70%), evaluation (15%), and testing (15%) sets for model development.

Key Insights and Ideas

  • Beyond Keywords: The study highlights the limitations of relying solely on predefined dictionaries (like LIWC) for bias detection. Fine-tuned language models offer a more powerful way to capture subtle linguistic nuances and their relationship to evaluation outcomes.
  • Day-to-Day Bias: By focusing on shift-based feedback, the research explores potential biases that might occur in the more frequent, less formal evaluations, which could be a critical early source of inequity.
  • Family and Children as a Theme: The finding that feedback for female students more frequently mentions family and children, even when not directly related to clinical performance, is a significant qualitative insight. This could reflect societal expectations or unconscious biases about women's roles.
  • No Systemic Score Bias: The absence of significant differences in the BERT model's prediction residuals between male and female students is a crucial finding, suggesting that, at this stage and with this dataset, overall performance scores are not systematically biased by gender.

Key Results

  • Residual Analysis: The study found no statistically significant difference in the prediction residuals between male and female students when using the BERT model. This indicates that the relationship between the text comments and the assigned global score was similar for both genders, suggesting no systematic group-level bias in how comments translate to scores.
  • LIWC Analysis: The LIWC analysis partially replicated previous findings, showing that comments given to female students contained a significantly higher percentage of social-communal themes (e.g., "families," "babies," "kids") compared to male students (p < 6 × 10⁻¹⁶). However, unlike some prior work, they did not find a significant difference in agentic terms or a significant difference in the "communal" theme (warmth/nurturing).
  • Overall Scores: While not finding bias in residuals, the study did note that female students received a slightly higher global score on average (mean difference of 0.08, p < 0.004). This is a marginal effect and does not contradict the finding of no bias in the translation of comments to scores.

Limitations and Future Work

  • Dataset Size and Scope: The authors acknowledge that the dataset is relatively small and limited to a specific type of evaluation (shift feedback in emergency medicine). Biases might be present in larger datasets or different evaluation contexts.
  • Noisy Mapping: The fine-tuned BERT model achieved relatively low accuracy in predicting the global score, indicating a "noisy mapping" between text and score in this dataset. This suggests that human judgment in these evaluations might be influenced by factors not fully captured by the text alone.
  • Gender as Binary: The study acknowledges the limitation of treating gender as a binary construct and hopes future work will explore a wider diversity of gender identification.
  • Future Directions: The authors suggest that future research should focus on collecting and analyzing similar data on a wider scale, exploring how bias manifests at different career stages, and combining multiple data sources to gain a more comprehensive view of bias and fairness. They also propose that the prediction residual method can be applied to other domains beyond medical education.

Summary updated

Abstract

Although approximately 50% of medical school graduates today are women, female physicians tend to be underrepresented in senior positions, make less money than their male counterparts and receive fewer promotions. There is a growing body of literature demonstrating gender bias in various forms of evaluation in medicine, but this work was mainly conducted by looking for specific words using fixed dictionaries such as LIWC and focused on recommendation letters. We use a dataset of written and quantitative assessments of medical student performance on individual shifts of work, collected across multiple institutions, to investigate the extent to which gender bias exists in a day-to-day context for medical students. We investigate differences in the narrative comments given to male and female students by both male or female faculty assessors, using a fine-tuned BERT model. This allows us to examine whether groups are written about in systematically different ways, without relying on hand-crafted wordlists or topic models. We compare these results to results from the traditional LIWC method and find that, although we find no evidence of group-level gender bias in this dataset, terms related to family and children are used more in feedback given to women.
BibTeX
@inproceedings{liu2022assessing,
  author = {Emmy Liu and Michael Henry Tessler and Nicole Dubosh and Katherine Hiller and Roger Levy},
  title = {{Assessing Group-level Gender Bias in Professional Evaluations: The Case of Medical Student End-of-Shift Feedback}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure