Abstract
Deep learning models often overfit in data-scarce, class-imbalanced scenarios typical of medical imaging. This paper demonstrates that such overfitting is asymmetric: while the majority class remains stable, the minority class "drifts" toward the decision boundary on unseen data, severely impacting sensitivity. We introduce asymmetric variants of popular training techniques—Large Margin Loss, Focal Loss, Adversarial Training, and Mixup—that prioritize the protection of the under-represented class. Our results show significant boosts in segmentation performance, particularly when training data is extremely limited.
Introduction
Convolutional Neural Networks (CNNs) are the standard for medical image segmentation but require substantial annotated data to generalize. In clinical practice, data is often scarce, and structures of interest (like tumors or small organs) frequently occupy less than 1% of the total volume. Standard regularization techniques often fail in these imbalanced settings. This research identifies that overfitting in these contexts manifests as a specific failure mode: precision remains high, but sensitivity plummets as the model becomes increasingly "shy" about predicting the minority class on new patients.
The Logit Shift Phenomenon
The core insight of this work is the "logit shift." By analyzing the raw scores (logits) before the final softmax layer, the authors found a stark difference between training and test distributions. For the majority background class, the logit distributions for training and test data are nearly identical. However, for the minority foreground class, the test activations shift significantly toward the decision boundary.
This suggests that while the network learns "generic" filters for the ubiquitous background, it "memorizes" the rare foreground samples using specific filters that fail to generalize. Consequently, the decision boundary, which appears safe during training, becomes too restrictive for unseen foreground structures.
Asymmetric Regularization Methods
To counteract the logit shift, the authors propose modifying standard machine learning "tricks" to be asymmetric, specifically "shielding" the minority class from the decision boundary.
- Asymmetric Large Margin Loss: Adds a penalty margin only to the minority class, forcing the network to learn more robust, distant features for it.
- Asymmetric Focal Loss: Down-weights "easy" samples only for the majority class, ensuring the network maintains focus on the minority class even if it appears easy during training.
- Asymmetric Adversarial Training: Generates difficult perturbed examples exclusively for the minority class to expand its representation in the feature space.
- Asymmetric Mixup: Blends images but assigns a "hard" minority label to any mix containing a significant portion of the rare class, preventing the boundary from encroaching on the foreground.
Experimental Evaluation
The methods were tested on two tasks: Brain Tumor Core Segmentation (BraTS) and Small Organ Segmentation (whole-body MRI). Data scarcity was simulated by using as little as 5% of the available training sets.
The asymmetric modifications consistently outperformed symmetric baselines. In the most extreme case (5% brain tumor data), Asymmetric Mixup achieved a Dice Similarity Coefficient (DSC) of 0.59, compared to 0.51 for standard Cross-Entropy. A combination of all four asymmetric techniques further boosted the DSC to 0.62, with sensitivity jumping from 0.43 to 0.66. Notably, while symmetric regularizers sometimes degraded performance compared to simple baselines, the asymmetric versions provided a consistent "safety net" and performance boost.
Conclusion
Symmetry in regularization can be detrimental in class-imbalanced segmentation. By acknowledging the asymmetric nature of logit drift, we can design more effective training protocols. Future work will explore extending these principles to multi-class segmentation and refining hyperparameter selection for these asymmetric losses.
Summary updated