BiSTF: Bilateral-Branch Self-Training Framework for Semi-Supervised Large-scale Fine-Grained Recognition

Hao Chang, Guochen Xie, Jun Yu, Qiang LingarXiv 2021 · arXiv

Abstract

Semi-supervised learning (SSL) often struggles with real-world complexities such as extreme class imbalance and the presence of out-of-class distractors in unlabeled data. We present the Bilateral-Branch Self-Training Framework (BiSTF), a novel approach designed for large-scale fine-grained recognition. BiSTF utilizes a dual-branch architecture to balance feature learning and a stochastic update strategy to mitigate the impact of noisy pseudo-labels. Evaluated on the Semi-iNaturalist benchmark, BiSTF demonstrates superior performance over existing SSL methods, particularly in handling long-tailed distributions and domain mismatches.

Introduction: The Challenges of Real-World SSL

Most semi-supervised learning (SSL) algorithms are developed using "clean" toy datasets where class distributions are perfectly balanced and unlabeled data belongs to the exact same categories as the labeled data. However, real-world data, exemplified by the Semi-iNaturalist (Semi-iNat) dataset, presents three major hurdles:

  1. Fine-Grained Similarity: Distinguishing between very similar species is difficult even with many labels.
  2. Class Imbalance: Models naturally bias toward majority classes, neglecting rare species.
  3. Domain Mismatch: The unlabeled pool often contains "out-of-class" images—species not present in the labeled set—which can "poison" the model with incorrect pseudo-labels.
Data distribution challenges in Semi-iNat: class imbalance and out-of-class noise.

Standard methods like FixMatch often fail in these scenarios because they lack mechanisms to handle such noise and imbalance.

The BiSTF Framework

The Bilateral-Branch Self-Training Framework (BiSTF) introduces a structured approach to overcome these challenges through three key innovations.

The BiSTF architecture featuring bilateral learning branches and a stochastic pseudo-label generation loop.

Bilateral-Branch Structure

Instead of a single training path, BiSTF employs two distinct branches:

  • In-class Learning Branch: Uses uniform sampling to learn general, discriminative features across all classes.
  • Semi-rebalancing Branch: Employs "reversed sampling" to focus on rare classes, counteracting majority-class bias. A Cumulative Learning strategy dynamically shifts the model's focus from general feature extraction to balanced classification as training progresses.

Stochastic Epoch Update

Traditional self-training updates pseudo-labels at fixed intervals. BiSTF introduces a stochastic approach: early in training, when the model is less reliable, updates are infrequent. As confidence and accuracy improve, the frequency of updates increases. This prevents the "garbage in, garbage out" cycle caused by early-stage noise.

Distribution Preservation

When selecting unlabeled images for pseudo-labeling, BiSTF ensures the resulting distribution matches the original labeled set. This "Contain Same Distribution" strategy prevents the model from amplifying its inherent bias toward majority classes during the self-training loop.

Experimental Setup and Datasets

The primary benchmark for this work is Semi-iNat, a large-scale dataset comprising roughly 330,000 images. It includes 810 in-class species and 1,629 out-of-class species acting as distractors. The dataset features a significant imbalance ratio of 16x, making it a rigorous test for SSL robustness.

Key Results and Insights

BiSTF achieves significant improvements over baseline SSL methods:

  • Performance Gains: BiSTF outperformed standard FixMatch by 1.79% absolute accuracy on the Semi-iNat private test set.
  • Backbone Efficiency: Utilizing EfficientNet with SENet blocks allowed the model to better attend to subtle fine-grained features. With an EfficientNet-b7 backbone, BiSTF reached a top-1 accuracy of 77.48%, a substantial leap from the official baseline.
  • Timing is Critical: Ablation studies confirmed that the "Linear" stochastic update strategy (gradually increasing update probability) is superior to fixed-interval or simple two-stage updates.

Conclusion and Future Work

BiSTF provides a robust framework for large-scale, imbalanced semi-supervised learning. While it shows strong results on Semi-iNat, future research could address the computational overhead of the bilateral structure and explore its generalization to other domains like medical or satellite imagery where out-of-class noise is prevalent.

Summary updated

BibTeX
@misc{chang2021bistf,
  author = {Hao Chang and Guochen Xie and Jun Yu and Qiang Ling},
  title = {{BiSTF: Bilateral-Branch Self-Training Framework for Semi-Supervised Large-scale Fine-Grained Recognition}},
  year = {2021},
  eprint = {2107.06768},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2107.06768},
}
Figure