Class imbalance remains a significant hurdle in deep learning, particularly in domains like medical imaging where rare conditions are underrepresented. Standard models trained with Softmax loss often prioritize majority classes, causing minority classes to be compressed into skewed regions of the feature space. This paper, presented at ICCV 2019, introduces Affinity Loss, a novel objective function designed to ensure that all classes—regardless of frequency—occupy distinct and well-defined territories in the feature space.
The Geometric Approach to Imbalance
Traditional Softmax loss relies on angular similarity (dot products), which lacks an explicit mechanism to enforce margins between classes. In imbalanced scenarios, majority classes naturally expand, "squeezing" the representation of rare classes.
Affinity Loss addresses this by shifting the optimization to Euclidean space. The method centers on three core components:
- Gaussian Affinity: Instead of angular alignment, the model measures the squared $\ell_2$ distance between a feature vector and a learned "class prototype" using a Gaussian similarity metric.
- Max-Margin Constraint: A hinge-loss penalty ensures that a sample's similarity to its ground-truth class is higher than its similarity to any other class by a predefined margin $\lambda$.
- Diversity Regularizer: This term forces class prototypes to be uniformly spaced and equidistant, preventing majority classes from crowding the feature space.
Experimental Results
The researchers evaluated Affinity Loss across diverse datasets, including the Dermofit Image Library (DIL) for skin lesions and the VGG2 dataset for face recognition.
Balanced Feature Representations
Visualizations of the learned feature space reveal the stark difference between standard approaches and Affinity Loss. While Softmax produces "floral petal" distributions where minority classes are thin and poorly separated, Affinity Loss generates clean, compact, and uniformly sized clusters for every category. This geometric regularity is key to the model's improved generalization on rare data.
Performance Benchmarks
- Skin Lesion Classification: On the DIL dataset, Affinity Loss achieved 91.1% accuracy, representing a 10.9% absolute improvement over the previous state-of-the-art (CoSen CNN).
- Robustness to Noise: The model demonstrated superior resilience to label noise. When labels were intentionally corrupted, Affinity Loss maintained significantly higher accuracy than Softmax, suggesting it learns robust underlying structures rather than simple label memorization.
- Face Recognition: Despite being a general-purpose loss, it achieved competitive results on high-stakes benchmarks, reaching 99.65% on LFW and 97.3% on YTF, outperforming specialized losses like Triplet Loss.
Conclusion and Future Directions
Affinity Loss successfully bridges the gap between classification and clustering. By enforcing a uniform geometric structure on the feature space, it prevents the model from becoming biased toward frequent classes. While the method requires careful tuning of hyperparameters like the cluster spread ($\sigma$) and margin ($\lambda$), it offers a powerful framework for robust learning in the presence of extreme class imbalance. Future work may explore automated methods for determining the optimal number of prototypes per class to better capture internal class variance.
Summary updated