Towards Cost-Effective Learning: A Synergy of Semi-Supervised and Active Learning

Tianxiang Yin, Ningzhong Liu, Han SunCVPR 2025 · DOI

Introduction

Deep learning's reliance on large labeled datasets presents a significant cost barrier. While Active Learning (AL) and Semi-Supervised Learning (SSL) offer solutions by selecting informative samples and leveraging unlabeled data respectively, their naive combination in Semi-Supervised Active Learning (SSAL) often underperforms. This paper introduces a novel Feature Re-alignment (FA) module to bridge this gap, addressing core issues like poor initial clustering and the "augmentation gap" that hinders uncertainty estimation.

Methodology: The Feature Re-alignment Module

The proposed FA module is a plug-and-play component for existing SSL frameworks (e.g., FixMatch, FreeMatch). It addresses the SSAL challenges through two primary mechanisms:

  1. Similarity Loss ($L_s$): To improve clustering when labels are extremely scarce, the module maintains a "feature queue" of labeled samples. Unlabeled samples are encouraged to align their weak-augmentation features with the most similar labeled feature in the queue, facilitating better group formation.
  2. Contrastive Loss ($L_c$): To stabilize uncertainty measurements for AL, this loss minimizes the discrepancy between features of weakly and strongly augmented versions of the same image. This ensures that the model's confidence scores remain reliable despite heavy distortions.
Overview of the Feature Re-alignment (FA) module integrating SSL and AL frameworks.

The authors theoretically ground this approach by decomposing SSAL risk into "pseudo error" (incorrect labels) and "margin error" (low confidence). They demonstrate that $L_s$ and $L_c$ directly target these error components.

Experimental Results

The FA module was evaluated on CIFAR-100 and TinyImageNet. The results demonstrate that the module consistently enhances performance across various SSL backbones and AL selection strategies.

Performance comparison on CIFAR-100 and TinyImageNet benchmarks showing significant gains with the FA module.

Key Findings:

  • Performance Boost: On CIFAR-100 with 100 labels, the FA module improved FreeMatch's accuracy from 53.44% to 60.93%.
  • AL Compatibility: The module makes AL strategies like AlpMix and ActFT more effective, consistently outperforming random selection—a common failure point in standard SSAL.
  • Clustering Quality: Visualizations confirm that the $L_s$ loss leads to tighter, more distinct feature clusters, which is essential for diversity-based AL.

Discussion and Future Work

While the FA module provides substantial gains, it introduces some computational overhead due to feature queue maintenance and additional loss calculations. Future research could explore the module's stability over multiple iterative AL rounds and its applicability to non-classification tasks like object detection or natural language processing.

Summary updated

BibTeX
@inproceedings{yin2025towards,
  author = {Tianxiang Yin and Ningzhong Liu and Han Sun},
  title = {{Towards Cost-Effective Learning: A Synergy of Semi-Supervised and Active Learning}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.00950},
}
Figure