Introduction: The Challenge of Label Noise
Training deep face recognition models on massive, real-world datasets is often hampered by significant label noise. In modern face recognition, the standard approach is to map images into a latent space where individuals are separated by "hyperplanes"—essentially, the model learns a central point or "anchor" for every person. To work well, these models require millions of images. However, datasets at this scale (like MS-Celeb-1M) are notoriously "noisy," containing many incorrectly labeled images.
The authors identify two specific types of conflict caused by this noise that create "gradient conflicts," preventing models from learning optimal features:
- Intra-class conflict: Different people are given the same label. This pulls the class anchor in contradictory directions, preventing the model from creating a tight cluster for that identity.
- Inter-class conflict: The same person is given two different labels. This forces the model to push apart features that should actually be close together.
When these conflicts occur, a single classification layer (hyperplane) cannot accurately describe the data, leading to unstable training and degraded accuracy.
Proposed Method: Switchable K-class Hyperplanes (SKH)
The core innovation is Switchable K-class Hyperplanes (SKH). Instead of the standard approach of using one weight matrix (one anchor per class), the authors propose using a mixture of $M$ independent hyperplanes.
- Greedy Switching: During training, for every image in a batch, the model calculates the loss against all $M$ hyperplanes. It then only updates the specific hyperplane that yielded the lowest loss. This allows the model to "assign" clean samples to one hyperplane while shunting noisy outliers to others, preventing the noise from polluting the gradients of the clean data.
- Posterior Data Clean (PDC): After the initial training phase, the authors use the learned anchors to refine the dataset. They identify anchors that have very few samples or are highly similar to anchors in other hyperplanes. By dropping high-confidence noise and merging split identities, they can retrain a final, "cleaner" model.
SKH is designed to be a drop-in replacement for standard classification layers and works seamlessly with popular margin-based loss functions like ArcFace and CosFace.
Key Insights and Results
The paper provides a mathematical derivation showing that noise doesn't just add "jitter"—it creates gradients perpendicular to the class centers, effectively "collapsing" the optimization process. By giving the model multiple "slots" (hyperplanes) to fit data into, the model naturally uses the extra slots to "trap" noise. Interestingly, the authors found that using just 3 hyperplanes ($M=3$) provided the best balance. Too few hyperplanes can't isolate the noise; too many can lead to "fragmented" identities where clean data is spread too thin.
The SKH method showed dramatic improvements, particularly on the noisiest datasets:
- Noise Robustness: On the noisy MS1MV0 dataset (original MS-Celeb-1M with ~50% noise), SKH + ArcFace achieved 93.50% (at FAR=1e-4) on IJB-B, compared to only 89.24% for vanilla ArcFace.
- State-of-the-Art: After applying the Posterior Data Clean (PDC) strategy, the model reached 94.82% on IJB-B and 96.26% on IJB-C, setting new benchmarks for noise-robust representation learning.
- Stability: In synthetic tests, as the noise ratio increased from 10% to 50%, the performance of standard ArcFace plummeted, while the SKH model remained remarkably stable.
Conclusion
While SKH increases memory and computation requirements during the training phase because multiple weight matrices must be maintained, it does not add any complexity during inference. The method demonstrates that allowing a model to choose between multiple hyperplanes is a more elegant and effective solution for handling massive label noise than manual pre-filtering. Future work could explore whether this "switchable" architecture helps in other domains with high label noise, such as large-scale product classification or medical imaging.
Summary updated