Switchable K-class Hyperplanes for Noise-Robust Representation Learning

Boxiao Liu, Guanglu Song, Manyuan Zhang, Haihang You, Yu Liu

Introduction: The Challenge of Label Noise

Training deep face recognition models on massive, real-world datasets is often hampered by significant label noise. In modern face recognition, the standard approach is to map images into a latent space where individuals are separated by "hyperplanes"—essentially, the model learns a central point or "anchor" for every person. To work well, these models require millions of images. However, datasets at this scale (like MS-Celeb-1M) are notoriously "noisy," containing many incorrectly labeled images.

The authors identify two specific types of conflict caused by this noise that create "gradient conflicts," preventing models from learning optimal features:

  1. Intra-class conflict: Different people are given the same label. This pulls the class anchor in contradictory directions, preventing the model from creating a tight cluster for that identity.
  2. Inter-class conflict: The same person is given two different labels. This forces the model to push apart features that should actually be close together.
Visualization of intra-class and inter-class conflicts caused by label noise.

When these conflicts occur, a single classification layer (hyperplane) cannot accurately describe the data, leading to unstable training and degraded accuracy.

Proposed Method: Switchable K-class Hyperplanes (SKH)

The core innovation is Switchable K-class Hyperplanes (SKH). Instead of the standard approach of using one weight matrix (one anchor per class), the authors propose using a mixture of $M$ independent hyperplanes.

  • Greedy Switching: During training, for every image in a batch, the model calculates the loss against all $M$ hyperplanes. It then only updates the specific hyperplane that yielded the lowest loss. This allows the model to "assign" clean samples to one hyperplane while shunting noisy outliers to others, preventing the noise from polluting the gradients of the clean data.
  • Posterior Data Clean (PDC): After the initial training phase, the authors use the learned anchors to refine the dataset. They identify anchors that have very few samples or are highly similar to anchors in other hyperplanes. By dropping high-confidence noise and merging split identities, they can retrain a final, "cleaner" model.
The SKH framework uses multiple independent hyperplanes to isolate noisy samples.

SKH is designed to be a drop-in replacement for standard classification layers and works seamlessly with popular margin-based loss functions like ArcFace and CosFace.

Key Insights and Results

The paper provides a mathematical derivation showing that noise doesn't just add "jitter"—it creates gradients perpendicular to the class centers, effectively "collapsing" the optimization process. By giving the model multiple "slots" (hyperplanes) to fit data into, the model naturally uses the extra slots to "trap" noise. Interestingly, the authors found that using just 3 hyperplanes ($M=3$) provided the best balance. Too few hyperplanes can't isolate the noise; too many can lead to "fragmented" identities where clean data is spread too thin.

The SKH method showed dramatic improvements, particularly on the noisiest datasets:

  • Noise Robustness: On the noisy MS1MV0 dataset (original MS-Celeb-1M with ~50% noise), SKH + ArcFace achieved 93.50% (at FAR=1e-4) on IJB-B, compared to only 89.24% for vanilla ArcFace.
  • State-of-the-Art: After applying the Posterior Data Clean (PDC) strategy, the model reached 94.82% on IJB-B and 96.26% on IJB-C, setting new benchmarks for noise-robust representation learning.
  • Stability: In synthetic tests, as the noise ratio increased from 10% to 50%, the performance of standard ArcFace plummeted, while the SKH model remained remarkably stable.

Conclusion

While SKH increases memory and computation requirements during the training phase because multiple weight matrices must be maintained, it does not add any complexity during inference. The method demonstrates that allowing a model to choose between multiple hyperplanes is a more elegant and effective solution for handling massive label noise than manual pre-filtering. Future work could explore whether this "switchable" architecture helps in other domains with high label noise, such as large-scale product classification or medical imaging.

Summary updated

Abstract

Optimizing the K-class hyperplanes in the latent space has become the standard paradigm for efficient representation learning. However, it’s almost impossible to find an optimal K-class hyperplane to accurately describe the latent space of massive noisy data. For this potential problem, we constructively propose a new method, named Switchable K-class Hyperplanes (SKH), to sufficiently describe the latent space by the mixture of K-class hyperplanes. It can directly replace the conventional single K-class hyperplane optimization as the new paradigm for noise-robust representation learning. When collaborated with the popular ArcFace on million-level data representation learning, we found that the switchable manner in SKH can effectively eliminate the gradient conflict generated by real-world label noise on a single K-class hyperplane. Moreover, combined with the margin-based loss functions (e.g. ArcFace), we propose a simple Posterior Data Clean strategy to reduce the model optimization deviation on clean dataset caused by the reduction of valid categories in each K-class hyperplane. Extensive experiments demonstrate that the proposed SKH easily achieves new state-of-the-art on IJB-B and IJB-C by encouraging noise-robust representation learning. Our code will be available at https://github.com/liubx07/SKH.git.
BibTeX
@inproceedings{liu2021switchable,
  author = {Boxiao Liu and Guanglu Song and Manyuan Zhang and Haihang You and Yu Liu},
  title = {{Switchable K-class Hyperplanes for Noise-Robust Representation Learning}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00301},
}
Figure