TL;DR
The researchers propose Debiased Noise Editing (DNE), a method to ensure fairness in medical image classification when using "black-box" Foundation Models (FMs) via APIs. By adding a learnable, minimal noise tensor to input images, DNE masks sensitive attributes (like gender) to prevent the model from relying on spurious correlations, achieving better fairness and utility without needing to modify the underlying FM.
What Problem Does This Paper Address?
Foundation Models (FMs) like GPT-4 or specialized medical models are increasingly used as feature extractors for healthcare diagnostics. However, these models often inherit and amplify biases present in their training data. In medical imaging, a model might inadvertently learn to identify a patient's gender or race from a chest X-ray and use that information as a shortcut for diagnosis—a phenomenon known as spurious correlation.
The challenge is that many state-of-the-art FMs are only accessible via APIs (black-boxes). Traditional bias mitigation techniques usually require retraining the model or having access to its internal gradients, which is impossible for web-hosted services like Google MedLM. This paper tackles the question: How can we remove bias from medical images when we can't change the model itself?
Methods
The authors introduce a pipeline that operates entirely on the input image space, making it model-agnostic.
- DNE (Debiased Noise Editing): Instead of changing the model, the authors learn a "noise" tensor $\epsilon$ with the same dimensions as the input image. This noise is optimized to "confuse" a pre-trained sensitive attribute (SA) classifier. By adding this minimal noise to the image, sensitive information is masked before the image is embedded by the FM.
- GeZO (Greedy Zeroth-Order Optimization): Since black-box APIs don't provide gradients, the authors developed GeZO. This optimization strategy estimates gradients by applying small, random perturbations to the noise and greedily selecting the direction that best reduces the sensitive information.
- Fair Inference: Once the DNE noise is learned, it is added to all new images. These "edited" images are then sent to the FM API to generate "fair embeddings," which are finally used to train a standard disease classifier.
Datasets
The study utilizes the CheXpert dataset, a large-scale collection of chest X-rays. The authors specifically focused on three binary classification tasks:
- Pleural Effusion
- Pneumonia
- Edema
They used gender (male vs. female) as the sensitive attribute. To rigorously test their method, they deliberately amplified the training data bias to see if DNE could still recover fair results.
Key Insights and Ideas
- Universal Utility: A single DNE noise pattern, once learned, can be applied across different disease classification tasks. This suggests that the noise effectively targets general sensitive features rather than task-specific ones.
- Interpretability through Visualization: By smoothing the DNE noise map, the authors could visualize which parts of the image the model was using to determine gender. As seen in Figure 3, the noise often targets areas like breast tissue or even text artifacts on the X-ray that might correlate with specific hospital protocols.
- Fairness-Utility Synergy: Surprisingly, DNE doesn't just improve fairness; it often improves accuracy. By forcing the model to ignore "shortcuts" (like gender), the model is pressured to find more robust, clinically relevant features for the actual disease.
Key Results
The empirical results demonstrate that DNE significantly outperforms traditional baselines in both fairness and accuracy.
- Fairness Gains: In the Pleural Effusion task, DNE reduced the Equal Opportunity (EO) disparity by over 25% compared to standard Empirical Risk Minimization (ERM) and other baselines like Group DRO.
- Accuracy Boost: DNE achieved an accuracy of 75.1% on Pleural Effusion, a 2.2% improvement over the baseline ERM (72.9%), indicating better generalization.
- Black-Box Effectiveness: The DNE-GeZO variant (for black-box APIs) performed comparably to the white-box version, proving that gradient access isn't strictly necessary for effective debiasing.
Limitations and Future Work
While promising, the authors note a few areas for further exploration:
- Broader Sensitive Attributes: This study focused primarily on gender. Future work needs to validate DNE against other sensitive attributes like race, age, and socioeconomic indicators.
- API Diversity: The method was tested using a ViT-base model to simulate an FM API. Testing on a wider variety of commercial black-box APIs (like MedLM or GPT-4V) would further prove its robustness.
- Hyperparameter Sensitivity: The regularization coefficient $\lambda$ and the number of local iterations $R$ in GeZO require careful tuning to balance image fidelity with debiasing strength.
Summary updated