Graph-Based Self-Training for Semi-Supervised Deep Similarity Learning

Yifan Wang, Yan Huang, Qicong Wang, Chong Zhao, Z. Zhang, Jian ChenSensors 2023 · DOI

TL;DR

This paper introduces a novel semi-supervised learning framework that enhances self-training by leveraging graph structures to better understand correlations between labeled and unlabeled data. By introducing an uncertainty-aware mechanism for generating more reliable pseudo-labels, the method significantly improves performance on tasks like image clustering and person re-identification.

What Problem Does This Paper Address?

Semi-supervised learning (SSL) is a powerful paradigm for training models when labeled data is scarce but unlabeled data is abundant. A common SSL technique is self-training, where a model is initially trained on labeled data, then uses its predictions to generate "pseudo-labels" for unlabeled data, and finally retrains on this augmented dataset. However, the effectiveness of self-training is heavily bottlenecked by the accuracy of these pseudo-labels. If the model generates incorrect pseudo-labels, especially with high confidence, it can lead the model astray, hindering performance. Existing methods often struggle to capture the nuanced relationships between labeled and unlabeled samples, and they may not effectively filter out noisy pseudo-labels. This paper aims to tackle these issues by improving both the quality of learned features and the reliability of pseudo-label generation.

Methods

The core of this work is a "positive and negative self-training framework based on graph-based deep uncertainty." This framework is built upon two key novel components:

  • Similarity Graph Structural Learning (SGSL) Model: This model explicitly learns a graph structure that captures the correlation between labeled and unlabeled samples within a batch. By considering these inter-sample relationships, SGSL helps the network learn more discriminative features, which in turn leads to more accurate predictions.
  • Uncertainty-Based Graph Convolutional Network (UGCN): UGCN leverages the graph structure learned by SGSL to aggregate neighborhood features, making similar features more cohesive. Crucially, UGCN also incorporates an uncertainty estimation mechanism. During pseudo-label generation, it uses dropout in a test-time manner to produce multiple predictions for the same sample. The standard deviation of these predictions serves as an uncertainty measure. This allows the framework to generate pseudo-labels only for unlabeled samples with low uncertainty, significantly reducing noise. Furthermore, the framework generates "negative pseudo-labels" for samples that are confidently predicted not to belong to a certain class, providing additional supervisory signals.

The overall framework operates in three stages: initial supervised pre-training, pseudo-label generation (using SGSL and UGCN with uncertainty filtering and negative pseudo-labeling), and iterative training with both labeled data and the newly generated pseudo-labeled data.

Datasets

The proposed method was evaluated on two types of tasks:

  • Image Clustering:
    • IJB-B (seven subsets, top 3 used: 512, 1024, 1845 identities)
    • IJB-C (four subsets, top 3 used: 1021, 1839, 3531 identities)
  • Person Re-Identification (Re-ID):
    • Market-1501
    • DukeMTMC-reID

For semi-supervised settings, only one-third of the training data was labeled, with the rest being unlabeled.

Key Insights and Ideas

  • Graph Structure for Unlabeled Data: The paper highlights that explicitly modeling the similarity graph between labeled and unlabeled data is crucial for improving feature discriminability in semi-supervised settings. This goes beyond simple feature extraction by incorporating relational information.
  • Uncertainty as a Noise Filter: Using prediction uncertainty (derived from repeated forward passes with dropout) is a robust way to identify and discard unreliable pseudo-labels, a common pitfall in self-training.
  • The Power of Negative Pseudo-Labels: Generating negative pseudo-labels for samples with low confidence provides valuable negative supervision, helping the model learn clearer decision boundaries and more discriminative features.
  • Synergy of SGSL and UGCN: The combination of SGSL for learning relationships and UGCN for uncertainty-aware feature aggregation and pseudo-labeling creates a powerful, end-to-end trainable system.

Key Results

  • Image Clustering: The proposed method achieved state-of-the-art or competitive results across multiple subsets of the IJB-B and IJB-C datasets, outperforming methods like K-Means, DBSCAN, ARO, and L-GCN. For instance, on IJB-B-512, it improved NMI by 2.89% over K-Means and 1.14% over L-GCN.
  • Person Re-ID: On the Market-1501 dataset, the proposed method achieved the best results across mAP, Rank-1, Rank-5, and Rank-10 metrics, outperforming TSSML by 0.8% in mAP and SPC-Combine by 3.6% in mAP. Similar competitive performance was observed on the DukeMTMC-reID dataset.
  • Ablation Study: Removing UGCN (variant 1) or the uncertainty mechanism (variant 2) led to performance degradation, underscoring the importance of both components. The proposed method (variant 3) showed significant improvements, e.g., +3.8% mAP over variant 1.
  • Runtime: The framework is efficient, with testing achieving approximately 1800 images per second, indicating near real-time capabilities.

Limitations and Future Work

  • Parameter Sensitivity: While the paper explores parameter sensitivity (e.g., threshold $\tau_p$), the optimal values might still require careful tuning for different datasets. The influence of $\tau_p$ was found to be relatively stable, but the best performance was achieved at $\tau_p = 0.8$.
  • Graph Convolutional Layer Depth: The performance was found to be relatively stable with respect to the number of graph convolutional layers, with two layers often yielding optimal results. Increasing depth did not consistently improve performance and could potentially lead to degradation.
  • Future Work: The authors suggest that their approach is well-suited for tasks requiring inter-sample similarity measurement, such as clustering and retrieval. The current work focuses on image data, but the graph-based nature of the method could potentially be extended to other data modalities.

Summary updated

Abstract

Semi-supervised learning is a learning pattern that can utilize labeled data and unlabeled data to train deep neural networks. In semi-supervised learning methods, self-training-based methods do not depend on a data augmentation strategy and have better generalization ability. However, their performance is limited by the accuracy of predicted pseudo-labels. In this paper, we propose to reduce the noise in the pseudo-labels from two aspects: the accuracy of predictions and the confidence of the predictions. For the first aspect, we propose a similarity graph structure learning (SGSL) model that considers the correlation between unlabeled and labeled samples, which facilitates the learning of more discriminative features and, thus, obtains more accurate predictions. For the second aspect, we propose an uncertainty-based graph convolutional network (UGCN), which can aggregate similar features based on the learned graph structure in the training phase, making the features more discriminative. It can also output the uncertainty of predictions in the pseudo-label generation phase, generating pseudo-labels only for unlabeled samples with low uncertainty; thus, reducing the noise in the pseudo-labels. Further, a positive and negative self-training framework is proposed, which combines the proposed SGSL model and UGCN into the self-training framework for end-to-end training. In addition, in order to introduce more supervised signals in the self-training process, negative pseudo-labels are generated for unlabeled samples with low prediction confidence, and then the positive and negative pseudo-labeled samples are trained together with a small number of labeled samples to improve the performance of semi-supervised learning. The code is available upon request.
BibTeX
@inproceedings{wang2023graph,
  author = {Yifan Wang and Yan Huang and Qicong Wang and Chong Zhao and Z. Zhang and Jian Chen},
  title = {{Graph-Based Self-Training for Semi-Supervised Deep Similarity Learning}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23083944},
}
Figure