ReRankMatch: Semi-Supervised Learning with Semantics-Oriented Similarity Representation

Trung Quang Tran, Mingu Kang, Daeyoung Kim

TL;DR

This paper introduces ReRankMatch, a semi-supervised learning (SSL) method that enhances the existing RankingMatch by incorporating "semantics-oriented similarity representation." This novel approach allows SSL to effectively handle scenarios where labeled and unlabeled data have non-overlapping categories, a common real-world challenge. ReRankMatch achieves state-of-the-art results on several benchmarks, demonstrating its effectiveness in leveraging unlabeled data for improved image classification.

What Problem Does This Paper Address?

Semi-supervised learning (SSL) methods typically assume that the unlabeled data used for training shares the same categories as the labeled data. This assumption is often violated in real-world applications, where unlabeled data might contain entirely different classes. For instance, if you have labeled images of cats and dogs, your unlabeled data might include cars and airplanes. Existing SSL methods struggle in such "non-overlapping category" scenarios. This paper tackles this limitation by proposing a method that can effectively learn from unlabeled data even when its categories are distinct from the labeled data.

Methods

ReRankMatch builds upon the RankingMatch framework, which combines consistency regularization and metric learning. The key innovation is the introduction of a semantics-oriented similarity representation. Here's how it works:

  • Feature Extractor & Classification Head: Like most deep learning models, ReRankMatch uses a feature extractor to generate image representations and a classification head to predict class scores (logits).
  • Class-Specific Part: This part is similar to RankingMatch. It uses labeled data to compute Cross-Entropy and Ranking losses. For unlabeled data, it leverages pseudo-labels generated from the model's predictions on weakly augmented versions.
  • Semantics-Oriented Part (The Novelty): This is where ReRankMatch shines.
  • Reference Images: From the labeled data, representative "reference" image representations are selected, one for each unique class present in the labeled set.
  • Similarity Representation: For each unlabeled image, its representation is compared against these reference images using cosine similarity. This generates a "semantics-oriented similarity representation" vector.
  • Pseudo-Labeling for Contrastive Loss: This similarity representation is then used to assign pseudo positive/negative labels for unlabeled data pairs. These labels help compute a feature contrastive loss. This loss encourages images from the same class (even if those classes are not in the labeled set) to have similar representations, and images from different classes to have dissimilar representations.

The overall objective is to minimize a combination of Cross-Entropy loss, Ranking loss, and this new feature contrastive loss.

Datasets

The paper evaluates ReRankMatch on several standard image classification benchmarks:

  • CIFAR-10: 10 classes, 32x32 images.
  • CIFAR-100: 100 classes, 32x32 images.
  • SVHN (Street View House Numbers): 10 classes, 32x32 images.
  • STL-10: 10 classes, 64x64 images.
  • Tiny ImageNet: 200 classes, 64x64 images.

The experiments vary the amount of labeled data used, from as few as 40 labels to 10,000 labels, to demonstrate the method's performance under different data scarcity conditions.

Key Insights and Ideas

  • Bridging the Category Gap: The core insight is that by learning a similarity representation based on any labeled data, even if those labels don't overlap with unlabeled data, the model can still leverage unlabeled data effectively. The semantics-oriented part learns to group similar-looking images together, regardless of their specific assigned label.
  • Semantics-Oriented Similarity is Crucial for Non-Overlapping Categories: The paper strongly suggests that this new component is what enables ReRankMatch to handle non-overlapping categories, a limitation of previous methods.
  • Data Sufficiency Matters: The performance boost from the semantics-oriented similarity representation is most pronounced when there's a "sufficient amount of labeled data" to learn meaningful representations. With very few labels, the benefit is less pronounced or even detrimental in some cases.
  • Cosine Similarity vs. Euclidean Distance: The paper notes a difference in cluster shapes in t-SNE visualizations between ReRankMatchBM (using Euclidean distance for logits) and ReRankMatchCT (using cosine similarity). This suggests that the choice of similarity metric for logits can impact learned representations.

Key Results

ReRankMatch demonstrates strong performance across various datasets and label regimes:

  • State-of-the-Art Achievements: ReRankMatch achieved new state-of-the-art results on CIFAR-10 (4.21% error rate with 4000 labels), CIFAR-100 (22.32% error rate with 10000 labels), and SVHN (2.19% error rate with 1000 labels).
  • Performance Gains with Few Labels: On CIFAR-10, ReRankMatchCT significantly outperformed RankingMatchBM when using only 40 or 250 labels, reducing error rates by up to 14.05% and 48.52% respectively.
  • ReRankMatchCT Superiority: The version using cosine similarity for logits (ReRankMatchCT) generally performed better than the Euclidean distance version (ReRankMatchBM), especially with fewer labels.
  • Meaningful Representation: The t-SNE visualizations show that ReRankMatch effectively clusters images by class, indicating that the learned representations are semantically meaningful.

Limitations and Future Work

  • Data Sufficiency: The paper acknowledges that the proposed semantics-oriented similarity representation is most beneficial when a "sufficient amount of labeled data" is available. The performance gain is less significant or even negative with very limited labeled data.
  • Future Work: The authors suggest further research to improve the performance of semantics-oriented similarity representation specifically in scenarios with "scanty" labeled data. They also plan to explore more advanced transformations for augmentation.

Summary updated

Abstract

This paper proposes integrating semantics-oriented similarity representation into RankingMatch, a recently proposed semi-supervised learning method. Our method, dubbed ReRankMatch, aims to deal with the case in which labeled and unlabeled data share non-overlapping categories. ReRankMatch encourages the model to produce the similar image representations for the samples likely belonging to the same class. We evaluate our method on various datasets such as CIFAR-10, CIFAR-100, SVHN, STL-10, and Tiny ImageNet. We obtain promising results (4.21% error rate on CIFAR-10 with 4000 labels, 22.32% error rate on CIFAR-100 with 10000 labels, and 2.19% error rate on SVHN with 1000 labels) when the amount of labeled data is sufficient to learn semantics-oriented similarity representation. The code is made publicly available at https://github.com/tqtrunghnvn/ReRankMatch.
BibTeX
@inproceedings{tran2021rerankmatch,
  author = {Trung Quang Tran and Mingu Kang and Daeyoung Kim},
  title = {{ReRankMatch: Semi-Supervised Learning with Semantics-Oriented Similarity Representation}},
  booktitle = {IJCNN},
  year = {2021},
  url = {https://openreview.net/forum?id=NIbo2nzGYO},
}

Loading related content...

Figure