Introduction
As Large Language Models (LLMs) become integral to healthcare—assisting in medical education and the generation of privacy-preserving synthetic datasets—the biases they harbor transition from theoretical concerns to matters of clinical safety. In the medical field, "stereotypical patients" often dominate literature, such as the middle-aged man being the face of cardiovascular disease. This study investigates how LLMs fine-tuned on French medical data replicate or amplify these societal biases, specifically focusing on gender representation in clinical narratives.
Methodology
The researchers developed a multi-step pipeline to quantify gender bias in a morphologically rich language like French, where nouns and adjectives inherently carry gender markers. Seven auto-regressive models (from the BioMistral, BLOOM, Vigogne, and Llama-3.1 families) were fine-tuned using Low-Rank Adaptation (LoRA) on a corpus of 2,048 real French clinical cases.
The models then generated a massive dataset of 21,000 synthetic clinical cases covering 10 specific disorders. To test for bias, the researchers used three prompt types: masculine, feminine, and neutral. A hybrid detection system, combining linguistic rules and machine learning, was used to classify the gender of the generated patients with 98% accuracy.
The Masculine Default and Gender Shift
A primary finding of the study is the existence of a "Masculine Default." When prompts were neutral and provided no gender information, models defaulted to male patients 1.9 times more often than female ones (64.4% vs. 33.5%). This suggests that within the models' latent space, the "standard" patient is male.
Furthermore, the study identified "Gender Shift," where models override explicit user instructions. For instance, in nearly 24% of cases involving breast cancer, models overridden masculine prompts to generate feminine markers, reflecting a strong association of the disease with women despite its occurrence in men. Conversely, models were less likely to shift feminine prompts toward masculine ones for diseases stereotypically associated with men.
Bias Amplification Across Disorders
The models do not merely reflect real-world imbalances; they significantly amplify them. While real-world data for heart attacks shows a roughly 60/40 male-to-female ratio, the generated cases exhibited an 8.1x disparity in favor of men when using neutral prompts.
As shown in the figure above, disorders like heart attacks, COVID-19, and colon cancer saw substantial increases in masculine representation compared to real-world prevalence. This amplification reinforces dangerous medical stereotypes that can lead to the underdiagnosis of women in clinical practice.
Key Results and Mitigation
- Model Variance: BLOOM-7b exhibited the highest gender gap, while Vigogne-2-13b was most effective at following gendered instructions (lowest Gender Shift).
- Mitigation Strategy: The most effective way to ensure a fair distribution of synthetic cases is to explicitly include demographic information in the prompt. Relying on the model to remain "neutral" consistently results in the least fair outcomes.
- Disorder Specificity: Bias is not uniform; while heart attacks showed extreme masculine bias, "neutral" disorders like colon cancer remained relatively more balanced, though still skewed.
Conclusion and Future Work
This research highlights that without explicit intervention, LLMs will continue to propagate and amplify gender biases in medical contexts. The authors acknowledge limitations, including a focus on binary gender due to French grammatical structures and the lack of a large-scale clinical review for medical plausibility. Future work will expand this methodology to other demographic markers, such as socioeconomic status and nationality, to further audit the fairness of clinical AI systems.
Summary updated