Abstract
Zero-Shot Learning (ZSL) faces a significant challenge known as "distribution shift," where the messy visual features of real-world data conflict with the idealized synthetic features generated for unseen classes. This paper proposes a novel framework that projects real visual features into a highly clusterable embedding space. By ensuring that real features mimic the well-separated nature of generated features, the model bridges the gap between seen and unseen distributions. The approach achieves state-of-the-art performance across several ZSL and Few-Shot Learning (FSL) benchmarks, demonstrating substantial gains in Generalized Zero-Shot Learning (GZSL) accuracy.
Introduction
Zero-Shot Learning (ZSL) aims to recognize objects from categories not encountered during training by leveraging semantic descriptions or attributes. Modern ZSL techniques often employ Conditional Variational Autoencoders (CVAEs) to synthesize artificial features for these unseen classes. However, a fundamental mismatch exists: real features extracted from backbones like ResNet are often noisy and lie near decision boundaries, whereas CVAE-generated features form tight, well-separated clusters. Training a classifier on "perfect" synthetic data and testing it on "messy" real data leads to poor generalization. This research addresses this by transforming real features to be more clusterable before the generative process begins.
Methodology
The proposed pipeline focuses on learning an orderly feature space through three primary components:
- Discriminative Embedding Space: A mapping function ($M$) projects original ResNet features into a new space supervised by a classification loss. This forces features from the same class to group together while pushing different classes apart.
- Gaussian Similarity Loss: Beyond standard cross-entropy, the model utilizes a Gaussian similarity loss (based on Radial Basis Functions) to explicitly optimize for "clusterability," enhancing intra-class compactness.
- Gaussian Noise Injection: To prevent the classifier from overfitting to perfectly clustered data, controlled Gaussian noise is injected into the projected features during CVAE training. This improves robustness by forcing the model to handle "harder" feature variants.
Experimental Setup
The method was evaluated on three standard ZSL benchmarks:
- CUB (Caltech-UCSD Birds-200-2011): 200 fine-grained bird species.
- SUN (Scene Attribute): 717 scene categories.
- AWA2 (Animals with Attributes 2): 50 animal classes with 85 attributes.
Additionally, the framework was tested on Few-Shot Learning (FSL) tasks using CUB and SUN to assess the versatility of the learned clusterable features.
Results and Discussion
The proposed method significantly outperformed previous state-of-the-art (SOTA) models, particularly in Generalized Zero-Shot Learning (GZSL), where both seen and unseen classes are present at test time.
Key Performance Gains
- Unseen Class Accuracy ($U$): The model achieved a 16.6% improvement on the SUN dataset and a 12.2% improvement on AWA2 compared to previous benchmarks.
- Conventional ZSL: Accuracy improved by 2.9% on CUB, 2.0% on SUN, and 2.8% on AWA2.
- Few-Shot Learning: In 1-shot scenarios, the method improved the baseline by 16.4% on CUB, highlighting the benefit of clusterable features in data-scarce regimes.
Insights
The core innovation lies in aligning real and synthetic distributions. By projecting real data into a space that mimics the clusterable nature of CVAE outputs, the distribution shift is minimized. The authors also found a strong correlation between feature "clusterability" (measured via Mutual Information) and final classification accuracy. Visualizations of the feature space confirm that while original ResNet features overlap significantly, the proposed projected features form distinct, easily separable islands.
Conclusion
This paper demonstrates that prioritizing clusterability in feature learning can effectively mitigate the distribution shift in ZSL. While the current work focuses on CVAE-based architectures, the "cluster-first" philosophy holds potential for other generative frameworks like GANs. Future research could extend this approach to other discriminative tasks such as face recognition or person re-identification, where separating highly similar identities is critical.
Summary updated