FairSeg: A Large-scale Medical Image Segmentation Dataset for Fairness Learning with Fair Error-Bound Scaling

Yu Tian, Min Shi, Yan Luo, Ava Kouhana, Tobias Elze, Mengyu Wang

In the rapidly evolving landscape of medical AI, "fairness" has transitioned from a theoretical concern to a clinical necessity. While significant progress has been made in identifying and mitigating biases in classification models, the field of medical image segmentation has largely been overlooked. This gap is critical, as segmentation provides the detailed spatial information clinicians rely on for diagnosing and treating complex diseases.

A research team from the Harvard Ophthalmology AI Lab recently presented FairSeg at ICLR 2024, introducing the first large-scale dataset and a novel algorithmic framework specifically designed to address fairness in medical image segmentation.

The Challenge of Fairness in Segmentation

Machine learning models often inadvertently learn biases from training data, leading to performance disparities across different races, genders, or age groups. In ophthalmology, for instance, Black individuals face a significantly higher risk of glaucoma, yet AI models frequently exhibit the lowest segmentation accuracy for this demographic.

The primary obstacles to achieving equity in this domain include:

  1. Data Scarcity: Most fairness-oriented datasets are tabular or focused on simple classification tasks.
  2. Annotation Difficulty: High-quality, pixel-wise medical annotations are labor-intensive and require expert knowledge.
  3. Metric Mismatch: Standard fairness metrics, such as demographic parity, are designed for discrete labels and do not easily translate to the continuous, spatial nature of segmentation masks.

Methods: FEBS and ESSP

The researchers addressed these challenges through a two-pronged approach involving a new training strategy and a specialized evaluation metric.

Fair Error-Bound Scaling (FEBS)

The core technical contribution is Fair Error-Bound Scaling (FEBS). Rather than treating all training errors equally, FEBS rescales the loss function using the upper error-bound of each identity group. This approach is based on the logic that disparities are often driven by "hard cases"—samples that are anatomically or physiologically distinct across demographics. By focusing on the highest-error samples within each group during training, the model is forced to learn features that are robust across all populations.

Equity-Scaled Segmentation Performance (ESSP)

To measure success, the authors introduced the ESSP metric (e.g., ES-Dice). This metric adjusts standard performance scores (like the Dice coefficient) by a factor representing the performance discrepancy across all groups. This creates a single, unified score that rewards both high accuracy and low demographic disparity.

The Harvard-FairSeg Dataset

The authors released the Harvard-FairSeg dataset, featuring 10,000 subject samples focused on the segmentation of the optic disc and cup—tasks critical for glaucoma diagnosis.

The Harvard-FairSeg data generation pipeline: 3D OCT B-scans are used to derive precise disc and cup borders, which are then registered to 2D SLO fundus images to create high-quality ground truth segmentation labels.

Uniquely, the labels were migrated from 3D OCT scans (a more accurate but expensive imaging modality) to 2D fundus images using automated registration and manual grading by five medical professionals. The dataset includes six sensitive attributes: age, gender, race, ethnicity, preferred language, and marital status, making it the most comprehensive dataset of its kind.

A comparison of Harvard-FairSeg with other prominent medical imaging datasets. Harvard-FairSeg is unique in its large scale, focus on 2D segmentation, and inclusion of a wide array of sensitive demographic attributes.

Key Results and Insights

The authors compared FEBS against state-of-the-art fairness methods like Adversarial Fair Representations (ADV) and Group Distributionally Robust Optimization (GroupDRO). Key findings include:

  • Superior Equity: On the "Rim" segmentation task—the most difficult part of the optic nerve to segment—the SAMed + FEBS model achieved the highest ES-Dice (0.7529) compared to the baseline SAMed (0.7478).
  • Consistency: FEBS consistently raised the "performance floor," improving accuracy for the lowest-performing groups (often Black or Asian subjects) without significantly degrading overall performance.
  • Language as a Proxy: Interestingly, the study found significant performance gaps when grouping by "Preferred Language," highlighting how socio-demographic factors beyond race and gender can impact AI performance.

Conclusion

By providing both the data and the tools to measure and mitigate bias, FairSeg sets a new standard for the development and evaluation of medical imaging AI. While currently focused on ophthalmology, the framework provides a blueprint for expanding fairness research into other domains like radiology and pathology, moving the field closer to true clinical equity.

Summary updated

BibTeX
@inproceedings{tian2024fairseg,
  author = {Yu Tian and Min Shi and Yan Luo and Ava Kouhana and Tobias Elze and Mengyu Wang},
  title = {{FairSeg: A Large-scale Medical Image Segmentation Dataset for Fairness Learning with Fair Error-Bound Scaling}},
  booktitle = {International Conference on Learning Representations},
  year = {2024},
  url = {https://openreview.net/forum?id=qNrJJZAKI3},
}
Figure