Introduction
Deep learning models in medical imaging face a critical dual challenge: catastrophic forgetting and shortcut learning. When models learn new tasks sequentially, they often overwrite previous knowledge. Simultaneously, they tend to latch onto "spurious correlations"—such as skin tone, patient age, or gender—rather than genuine pathological features. Existing Continual Learning (CL) methods often preserve or even amplify these biases. BiasPruner addresses this by introducing "intentional forgetting," selectively pruning network units that learn these biased shortcuts to create fair, robust subnetworks.
Methodology: Intentional Forgetting
BiasPruner operates on the principle that neural networks are "lazy," fitting easy, biased samples before complex clinical features. The framework follows a three-stage pipeline:
- Bias Scoring: The model is initially trained with Generalized Cross Entropy (GCE) loss. Units are scored based on activation differences between "easy" samples (likely biased) and "hard" samples (likely unbiased).
- Pruning and Fine-tuning: Units with high bias scores are pruned. The remaining subnetwork is fine-tuned using Weighted Cross Entropy (WCE) to emphasize hard samples, forcing the model to learn clinically relevant features.
- Debiased Knowledge Transfer: For sequential tasks, BiasPruner allocates specific, frozen subnetworks for old tasks while utilizing remaining capacity for new ones, preventing both forgetting and bias transfer.
For inference, the model employs a task-agnostic "maxoutput" strategy, automatically selecting the subnetwork with the strongest response without requiring task-specific labels.
Experimental Evaluation
The method was evaluated on three benchmarks with known sensitive attributes:
- FITZ (Fitzpatrick17K): Skin lesion classification (6 tasks) biased by skin tone.
- HAM (HAM10000): Skin lesion classification (3 tasks) biased by patient age.
- NIH (ChestX-Ray14): Thorax disease classification (3 tasks) biased by patient gender.
Notably, BiasPruner does not require explicit labels for these sensitive attributes during training, making it highly suitable for privacy-constrained medical environments.
Key Results and Discussion
BiasPruner consistently outperformed state-of-the-art CL methods in both diagnostic accuracy and fairness metrics. On the FITZ dataset, it achieved an overall balanced accuracy of 0.512, significantly higher than standard CL methods like EWC (0.324).
Beyond accuracy, BiasPruner significantly improved fairness. It achieved the lowest Equal Opportunity Difference (EOD) of 0.202 on FITZ, indicating more equitable performance across demographic groups. Probing experiments confirmed that the model's internal features were less predictive of sensitive attributes, verifying that the pruning effectively "unlearned" the bias rather than just hiding it.
Conclusion
BiasPruner demonstrates that forgetting can be a feature rather than a bug in medical AI. By selectively removing biased network components, the framework enables the development of models that are both continually learnable and fundamentally fairer. Future work will explore scaling this pruning logic to larger architectures like Vision Transformers and medical foundation models.
Summary updated