Semi-supervised Active Learning for Semi-supervised Models: Exploit Adversarial Examples with Graph-based Virtual Labels

Jiannan Guo, Haochen Shi, Yangyang Kang, Kun Kuang, Siliang Tang, Zhuoren Jiang, Changlong Sun, Fei Wu, Yueting Zhuang

Deep learning models are notoriously data-hungry, yet high-quality human annotation remains expensive and slow. To mitigate this, researchers typically employ two strategies: Active Learning (AL), which identifies the most informative samples for manual labeling, and Semi-supervised Learning (SSL), which leverages unlabeled data alongside a small labeled set.

Presented at CVPR 2021, this paper introduces REVIVAL (gRaph-basEd VIrtual adVersarial Active Learning), a framework designed to bridge the gap between these two paradigms. REVIVAL creates a "closed-loop" where SSL and AL mutually enhance one another, significantly reducing the manual annotation required for state-of-the-art performance.

The Mismatch Problem

The authors identify a critical "mismatch problem" in existing hybrid approaches. Many current methods, such as VAAL, utilize generative models (like VAE-GANs) to learn latent representations based solely on whether a sample is "labeled" or "unlabeled," often ignoring actual class labels.

The mismatch problem in latent space (top) versus the class-aware latent space achieved by REVIVAL (bottom).

As illustrated above, this lack of class awareness often results in samples from different classes (e.g., a cat and a fish) being projected into the same area in latent space. Furthermore, many AL methods fail to utilize SSL for the final task model, essentially wasting the unlabeled data analyzed during the selection process.

The REVIVAL Framework

REVIVAL addresses these issues through three integrated modules that work in a continuous cycle to ensure the latent space respects class boundaries while identifying the most informative samples.

The REVIVAL framework architecture, detailing the interaction between the Label Propagator, Virtual Adversarial Generator, and Boundary Limitator.

1. The Label Propagator (LP)

This module constructs a sparse k-Nearest Neighbor (k-NN) graph of all samples based on their feature representations. It employs a Graph Convolutional Network (GCN) to propagate label information, assigning "virtual labels" to unlabeled data. This captures the underlying structural distribution of the dataset.

2. The Virtual Adversarial Generator (VAG)

To identify high-value samples, the VAG searches for data points near cluster boundaries. It applies small adversarial perturbations to unlabeled samples and measures the resulting change in the model's prediction using KL-divergence. If a minor nudge in feature space causes a significant shift in the virtual label, the sample is likely on a decision boundary and highly informative.

3. The Boundary Limitator (BL)

From the candidates identified by the VAG, the BL selects final samples for human annotation based on the entropy of their virtual labels, prioritizing instances where the model's uncertainty is highest.

Experimental Results

The authors evaluated REVIVAL on standard benchmarks including CIFAR-10, CIFAR-100, and Cityscapes (for semantic segmentation).

  • Superior Efficiency: On CIFAR-10, REVIVAL achieved 93.5% accuracy using only 35% of the labeled data, consistently outperforming state-of-the-art baselines like VAAL, LLAL, and Core-set.
  • Segmentation Generality: In the Cityscapes task, REVIVAL reached a target mean Intersection over Union (mIoU) of 58.9% with 10% less labeled data than competing methods.
  • Ablation Studies: Removing any single component (LP, VAG, or BL) led to significant performance drops, confirming that the synergy between graph structure and adversarial searching is critical to the framework's success.

Conclusion

REVIVAL redefines the relationship between Active Learning and Semi-supervised Learning. By treating AL as a "graph rectifier" that clarifies ambiguous boundaries, the framework ensures that manual annotations provide maximum benefit to the SSL model's structural understanding. While computational overhead for massive datasets remains a consideration for future work, REVIVAL offers a powerful path toward more label-efficient computer vision training.

Summary updated

Abstract

The performance of computer vision models significantly improves with more labeled data. However, the acquisition of labeled data is limited by the high cost. To mitigate the reliance on large labeled datasets, active learning (AL) and semi-supervised learning (SSL) are frequently adopted. Although current mainstream methods begin to combine SSL and AL (SSL-AL) to excavate the diverse expressions of unlabeled samples, these methods’ fully supervised task models are still trained only with labeled data. Besides, these method’s SSL-AL frameworks suffer from mismatch problems. Here, we propose a graph-based SSL-AL framework to unleash the SSL task models’ power and make an effective SSL-AL interaction. In the framework, SSL leverages graph-based label propagation to deliver virtual labels to unlabeled samples, rendering AL samples’ structural distribution and boosting AL. AL finds samples near the clusters’ boundary to help SSL perform better label propagation by exploiting adversarial examples. The information exchange in the closed-loop realizes mutual enhancement of SSL and AL. Experimental results show that our method outperforms the state-of-the-art methods against classification and segmentation benchmarks.
BibTeX
@inproceedings{guo2021semi,
  author = {Jiannan Guo and Haochen Shi and Yangyang Kang and Kun Kuang and Siliang Tang and Zhuoren Jiang and Changlong Sun and Fei Wu and Yueting Zhuang},
  title = {{Semi-supervised Active Learning for Semi-supervised Models: Exploit Adversarial Examples with Graph-based Virtual Labels}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00289},
}
Figure