Abstract
Standard image classification models typically rely on the "smoothness assumption," where visual similarity implies shared labels. However, real-world tasks often require grouping visually heterogeneous objects into broad "superclasses" (e.g., refuse sorting). We introduce Superclass Learning with Representation Enhancement (SCLRE), a framework that leverages Cross-Instance Attention (CIA) and a novel tripartite loss function to discover semantic links across visually diverse instances. Our results demonstrate that SCLRE significantly improves classification accuracy in superclass scenarios across multiple benchmarks, including CIFAR-100 and real-world datasets like FMoW.
Introduction
Most computer vision models excel at basic-level classification, such as distinguishing between specific species or objects. These models operate under the assumption that images with similar visual features should reside near each other in the embedding space. This assumption fails in the context of Superclasses. For instance, in waste management, "kitchen waste" encompasses items as visually distinct as eggshells, bones, and rotten fruit. Conversely, a "toy apple" (recyclable) may be visually indistinguishable from a "real apple" (kitchen waste).
Traditional models struggle because sub-categories within a superclass are often scattered, and boundaries are blurred by visual mimics. SCLRE addresses this by shifting focus from individual image features to relational semantic features within a data batch.
Methodology
The SCLRE framework moves beyond isolated image processing by implementing two primary components:
Cross-Instance Attention (CIA)
Instead of extracting features from a single image, SCLRE employs a self-attention mechanism across an entire data batch. This allows the model to "mix" representations, facilitating the discovery of high-level semantic links between visually disparate items (e.g., a bone and an eggshell) that share a superclass label.
Tripartite Loss Strategy
To reshape the embedding space effectively, SCLRE utilizes three loss functions:
- Category Classification Loss ($L_{ce}$): Standard cross-entropy to maintain baseline predictive accuracy.
- Contrastive Adjustment Loss ($L_{ca}$): Encourages representations of the same superclass to cluster while pushing different superclasses apart.
- Targeted Adjustment Loss ($L_{ta}$): This key innovation defines fixed "target anchors" (category centers) in the embedding space. Samples are forced to migrate toward these anchors, preventing the "scattered" distribution typical of complex superclasses.
Experimental Evaluation
The authors evaluated SCLRE on several benchmarks reorganized to simulate superclass challenges:
- CIFAR-100: Grouped into 3, 4, and 7 superclasses (e.g., "mammals," "household items").
- High-Resolution Benchmarks: mini-ImageNet and VOC.
- Real-World Datasets: Functional Map of the World (FMoW) for regional building style prediction and Adience for age group estimation.
Results and Discussion
SCLRE consistently outperformed baselines and standard contrastive learning methods. On CIFAR100-3, SCLRE achieved 80.1% accuracy, surpassing the ResNet baseline (72.8%) and SupCon (78.1%). Notably, on the complex FMoW dataset, SCLRE improved performance by over 8% (64.8% vs 56.5%), demonstrating robustness in non-idealized scenarios.
Visualizations using t-SNE confirmed that while baseline models produce scattered "clouds" for superclasses, SCLRE generates tight, well-separated clusters. This suggests that the combination of CIA and targeted anchors successfully collapses sub-category boundaries in favor of semantic super-structures.
Conclusion
SCLRE provides a theoretically grounded and practically effective approach to superclass learning. By breaking the reliance on individual visual features and utilizing batch-level attention, the model captures the underlying semantic essence of broad categories. Future work will focus on reducing batch dependency and exploring semi-supervised methods to discover these super-structures automatically without extensive expert labeling.
Summary updated