Abstract
This paper investigates the problem of semi-supervised classification. Unlike previous methods to regularize classifying boundaries with unlabeled data, our method learns a new image representation from all available data (labeled and unlabeled) and performs plain supervised learning with the new feature. In particular, an ensemble of image prototype sets are sampled automatically from the available data, to represent a rich set of visual categories/attributes. Discriminative functions are then learned on these prototype sets, and image are represented by the concatenation of their projected values onto the prototypes (similarities to them) for further classification. Experiments on four standard datasets show three interesting phenomena: (1) our method consistently outperforms previous methods for semi-supervised image classification, (2) our method lets itself combine well with these methods, and (3) our method works well for self-taught image classification where unlabeled data are not coming from the same distribution as labeled ones, but rather from a random collection of images.
Introduction
Semi-supervised learning (SSL) aims to enhance image classifiers by utilizing a small amount of labeled data alongside a large pool of unlabeled data. Traditional approaches typically focus on "regularizing" decision boundaries, often relying on local consistency assumptions—where similar images are expected to share labels. However, these methods frequently ignore global information from dissimilar images, require complex specialized algorithms, and struggle when unlabeled data distributions do not perfectly match labeled categories (the "self-taught learning" scenario).
This paper introduces Ensemble Projection (EP), a framework that reframes SSL as a feature-learning task rather than a boundary-regularization problem. By learning a high-level representation from all available data, EP allows standard "off-the-shelf" classifiers to achieve state-of-the-art results.
Methodology
The Ensemble Projection pipeline transforms raw image descriptors into a discriminative feature space through two primary stages:
1. Max-Min Sampling
To build a robust feature space, the system identifies "prototypes"—representative clusters of visual concepts—from the unlabeled data.
- Max Step: The algorithm identifies diverse visual concepts by sampling sets of images and selecting those with maximum mutual distance. This ensures the prototypes cover a wide range of visual attributes (e.g., sky, grass, urban structures).
- Min Step: For each seed image identified, the algorithm aggregates its $n$ nearest neighbors to form a compact and robust cluster.
2. Feature Projection
Once prototype sets are established, $T$ ensemble classifiers (typically simple Logistic Regression) are trained. For any input image, a new feature vector is generated by concatenating its similarity scores—or "projections"—across all these prototypes. This process leverages exotic-inconsistency: the principle that highly dissimilar samples likely belong to different classes. The resulting projection vector captures high-level semantic attributes that low-level descriptors like GIST or PHOG often miss.
Experimental Evaluation
The authors evaluated EP on four benchmarks: Scene-15 (indoor/outdoor scenes), LandUse-21 (satellite imagery), Texture-25 (textures), and Caltech-101 (object recognition).
Performance and Complementarity
A key finding is that EP-based features significantly boost the performance of standard classifiers. For instance, on Scene-15 with only 5 labeled examples per class, Logistic Regression using EP features achieved a Mean Average Precision (MAP) of 61.3%, compared to 38.6% with standard features.
Furthermore, EP is complementary to existing SSL methods. When EP features are used as input for algorithms like Harmonic-Function (HF) or Laplacian SVM (LSVM), performance improves further, indicating that EP captures unique information about the data manifold.
Self-Taught Learning
The method also excels in self-taught learning scenarios. By using 20,000 random images from ImageNet as the unlabeled "universal visual world," the authors demonstrated that EP can learn useful visual primitives (colors, shapes, textures) that generalize to specific classification tasks, even when the unlabeled data distribution is entirely different from the target classes.
Conclusion
Ensemble Projection shifts the focus of semi-supervised learning from complex boundary optimization to effective feature engineering. By automatically discovering and projecting onto visual prototypes, it provides a flexible, robust, and high-performing representation for image classification. While the approach introduces some computational overhead during the ensemble training phase, its parallelizability and compatibility with standard classifiers make it a powerful tool for real-world computer vision applications.
Summary updated