Abstract
This paper introduces Appearance-Based Curriculum (ABC), a novel semi-supervised learning framework designed for scenarios where labeled data is restricted to specific viewpoints (like front-facing catalog images) while unlabeled data comes from diverse, unconstrained angles. By incrementally feeding unlabeled data into the training process—starting with images most similar to the labeled set—ABC maintains high pseudo-label quality and significantly outperforms state-of-the-art methods like FixMatch in multi-angle recognition tasks.
Introduction: The Challenge of Viewpoint Shift
In many practical computer vision applications, there is a massive distribution shift between labeled and unlabeled data. Consider product recognition: a company might have a "digital catalog" with one or two perfectly lit, front-facing labeled images per product. However, the unlabeled data collected from in-store security cameras features the same products from every possible angle, often obscured or rotated.
Standard Semi-Supervised Learning (SSL) algorithms, such as FixMatch or VAT, typically assume that unlabeled data follows the same distribution as labeled data. They rely on data augmentation to bridge gaps, but standard augmentations (like flipping or cropping) cannot transform a front-facing photo into a view from the back. When these models are trained on all unlabeled data simultaneously, they generate "noisy" (incorrect) pseudo-labels for the difficult angles, which eventually degrades the model's performance.
The Appearance-Based Curriculum (ABC) Framework
The authors propose a two-phase framework called Appearance-Based Curriculum (ABC) to bridge the gap between catalog photos and real-world views.
- Curriculum Construction: The unlabeled data is ranked from "easy" to "hard" based on its similarity to the labeled set.
- Training Phase: Instead of using all unlabeled data at once, the model (using a base SSL algorithm like FixMatch) is trained in steps. It starts with the most similar unlabeled images. Once the model reaches a performance plateau (the "Feed-on-Plateau" criterion), the next batch of slightly more "difficult" (dissimilar) images is added to the training pool.
This "easy-to-hard" approach ensures that the model first learns to classify images that look like the catalog photos, then uses that knowledge to accurately pseudo-label images from slightly different angles, gradually working its way around the object.
Distance Functions for Ranking
The authors propose three distance functions ($d$) to measure the similarity between labeled and unlabeled data, allowing the curriculum to be constructed effectively:
- $d_{angle}$: Uses actual camera angle metadata (elevation and azimuth) if available.
- $d_{local}$: Uses hand-crafted local descriptors (AKAZE) to match keypoints between images. This works well when objects have distinct textures, such as beverage labels.
- $d_{global}$: Uses a feature extractor pretrained on the small labeled set to compute cosine similarity between global image features.
Experimental Evaluation
The researchers evaluated ABC on three object recognition benchmarks: MIRO (120 objects, 160 angles), COIL-100 (100 objects, 72 angles), and DRINK, a newly introduced private dataset of 65 Japanese beverage brands.
Key Results
The results demonstrate that ABC provides a massive boost in accuracy over existing state-of-the-art SSL methods:
- MIRO Dataset: ABC ($d_{global}$) achieved up to 86.5% accuracy, whereas the base FixMatch algorithm struggled at 65.4%.
- DRINK Dataset: ABC ($d_{local}$) reached 81.4% accuracy, nearly doubling the performance of the supervised-only baseline (46.4%) and significantly beating FixMatch (48.3%).
- Robustness: ABC showed it could handle various starting angles for the labeled data, consistently outperforming other methods regardless of whether the "front" or "top" of the object was labeled first.
Conclusion
ABC avoids the "pseudo-label quality trap" by "growing" the model's expertise. By the time the model sees a side-view, it has already learned from "near-front" views, making the side-view easier to classify. While the success of ABC relies heavily on choosing an appropriate distance function and involves some computational overhead due to multi-stage training, it offers a robust solution for real-world distribution shifts. Future work may investigate if this curriculum approach can be applied to other shifts, such as lighting or background changes.
Summary updated