Deep Co-Space: Sample Mining Across Feature Transformation for Semi-Supervised Learning

Ziliang Chen, Keze Wang, Xiao Wang, Pai Peng, Ebroul Izquierdo, Liang Lin

Abstract

Aiming at improving performance of visual classification in a cost-effective manner, this paper proposes an incremental semi-supervised learning paradigm called Deep Co-Space (DCS). Unlike many conventional semi-supervised learning methods usually performing within a fixed feature space, our DCS gradually propagates information from labeled samples to unlabeled ones along with deep feature learning. We regard deep feature learning as a series of steps pursuing feature transformation, i.e., projecting the samples from a previous space into a new one, which tends to select the reliable unlabeled samples with respect to this setting. Specifically, for each unlabeled image instance, we measure its reliability by calculating the category variations of feature transformation from two different neighborhood variation perspectives, and merged them into an unified sample mining criterion deriving from Hellinger distance. Then, those samples keeping stable correlation to their neighboring samples (i.e., having small category variation in distribution) across the successive feature space transformation, are automatically received labels and incorporated into the model for incrementally training in terms of classification. Our extensive experiments on standard image classification benchmarks (e.g., Caltech-256 and SUN-397) demonstrate that the proposed framework is capable of effectively mining from large-scale unlabeled images, which boosts image classification performance and achieves promising results compared to other semi-supervised learning methods.

Introduction

Deep learning models typically require massive amounts of labeled data to achieve high performance. However, obtaining high-quality labels is often expensive and time-consuming. Semi-supervised learning (SSL) offers a solution by leveraging large pools of unlabeled data alongside a limited labeled set. A significant challenge in SSL is "semantic drift," where a model becomes increasingly confident in its own erroneous predictions during iterative training, leading to a degradation in performance.

Traditional SSL methods often operate within a fixed feature space, failing to account for the dynamic nature of feature learning in deep neural networks. This paper introduces Deep Co-Space (DCS), an incremental paradigm that identifies reliable unlabeled samples by observing how their local neighborhood structures evolve across successive feature transformations.

Methodology

The DCS framework treats deep feature learning as a sequence of transformations that project data into evolving feature spaces. The core innovation lies in mining unlabeled samples that demonstrate "stability" during these transitions.

Overview of the Deep Co-Space (DCS) framework, illustrating the iterative process of sample mining and model updating.

The Co-Space Concept

A "Co-Space" is defined by two successive feature spaces: one generated by the model at iteration $t-1$ and the other at iteration $t$. By analyzing the relationship between an unlabeled sample and its labeled neighbors across these two spaces, DCS can gauge the reliability of a potential pseudo-label.

Illustration of feature space transformation. DCS identifies stable unlabeled samples by comparing their local neighborhoods in successive feature spaces.

Sample Mining Criterion

DCS evaluates reliability through a dual-variation criterion:

  1. Neighborhood Intrinsic Variation: This measures whether the predicted soft labels for a sample (derived via Label Propagation on a K-nearest neighbor graph) remain consistent across the transformation.
  2. Neighborhood Category Variation: This uses Hellinger distance to measure changes in local density and class-specific statistics, ensuring the sample's relationship to known categories remains stable.

To handle high-dimensional CNN features effectively and avoid the "curse of dimensionality," the framework incorporates LargeVis for dimensionality reduction before graph-based analysis.

Experimental Evaluation

The framework was evaluated on the Caltech-256 and SUN-397 benchmarks. For Caltech-256, the authors supplemented the dataset with ~25,600 noisy web-crawled images to simulate a large-scale unlabeled pool.

Key Results

  • Performance Gains: On Caltech-256, DCS significantly improved classification. For example, using a VGG-based architecture with 18.8% labeled data, DCS reduced the error rate of a Stacked What-Where Auto-Encoder (SWWAE) from 29.42% to 26.73%.
  • Superiority over Baselines: On SUN-397, DCS achieved 74.56% accuracy with 50% labeling, outperforming popular SSL techniques like Deep Semi-Supervised Embedding (DSSE) and even the fully supervised baseline (73.07%).
  • Drift Resistance: Ablation studies confirmed that the Co-Space transformation analysis is vital. Without it, the accuracy of pseudo-labeled samples dropped rapidly. With DCS, the model maintained over 80% label accuracy for mined samples across multiple iterations, effectively halting semantic drift.

Conclusion

Deep Co-Space provides a robust mechanism for incremental semi-supervised learning by focusing on the stability of local data structures during feature learning. By combining intrinsic and category-based variation metrics, DCS effectively mitigates semantic drift and boosts classification performance. Future work may explore extending this stability-based mining to more complex visual tasks such as object detection and semantic segmentation.

Summary updated

Abstract

Aiming at improving performance of visual classification in a cost-effective manner, this paper proposes an incremental semi-supervised learning paradigm called Deep Co-Space (DCS). Unlike many conventional semi-supervised learning methods usually performing within a fixed feature space, our DCS gradually propagates information from labeled samples to unlabeled ones along with deep feature learning. We regard deep feature learning as a series of steps pursuing feature transformation, i.e., projecting the samples from a previous space into a new one, which tends to select the reliable unlabeled samples with respect to this setting. Specifically, for each unlabeled image instance, we measure its reliability by calculating the category variations of feature transformation from two different neighborhood variation perspectives, and merged them into an unified sample mining criterion deriving from Hellinger distance. Then, those samples keeping stable correlation to their neighboring samples (i.e., having small category variation in distribution) across the successive feature space transformation, are automatically received labels and incorporated into the model for incrementally training in terms of classification. Our extensive experiments on standard image classification benchmarks (e.g., Caltech-256 and SUN-397) demonstrate that the proposed framework is capable of effectively mining from large-scale unlabeled images, which boosts image classification performance and achieves promising results compared to other semi-supervised learning methods.
BibTeX
@article{chen2017deep,
  author = {Ziliang Chen and Keze Wang and Xiao Wang and Pai Peng and Ebroul Izquierdo and Liang Lin},
  title = {{Deep Co-Space: Sample Mining Across Feature Transformation for Semi-Supervised Learning}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2017},
  doi = {10.1109/tcsvt.2017.2710478},
}
Figure