Abstract
Zero-shot learning (ZSL) aims to recognize novel object categories using the semantic representation of categories, exploring how novel classes are semantically related to familiar ones. Key models learn embeddings between image feature space and semantic space, but it is crucial to learn discriminative features that comprise coarse-to-fine information. We propose a discriminative embedding autoencoder with a regressor feedback model for ZSL. The encoder learns a mapping to a discriminative embedding space, regulating inter-class and intra-class distances via triplet loss. The regressor feedback maps reconstructed samples back to the embeddings, assisting the decoder to improve sample quality and generalization. Validated on SUN, CUB, AWA1, and AWA2 datasets, our model outperforms state-of-the-art methods, particularly in generalized zero-shot learning (GZSL).
Introduction
The core challenge of Zero-Shot Learning (ZSL) is teaching a machine to recognize objects it has never seen before (e.g., identifying a "zebra" after only being trained on "horses" and "tigers"). This is typically achieved by mapping visual features to a shared semantic space, such as a list of attributes like "has stripes" or "four legs."
However, two major hurdles persist in current research:
- Projection Domain Shift: Models trained only on "seen" classes often become biased, causing "unseen" class projections to cluster near seen ones, leading to misclassification.
- Lack of Discriminative Power: Standard semantic attributes are often too coarse to distinguish between visually similar classes.
Proposed Methodology
The proposed architecture is designed as a sophisticated autoencoder consisting of three primary components:
- The Discriminative Encoder: Instead of a simple mapping, the encoder utilizes triplet loss. This forces the model to cluster images of the same class together while pushing different classes apart by a specific margin, ensuring the latent embedding space is inherently discriminative.
- The Decoder (Generator): The decoder takes a concatenation of the discriminative embedding (from the encoder) and the semantic embedding (attributes) to reconstruct the original image features. This dual-input approach ensures the generator has both high-level category information and fine-grained visual data.
- The Regressor Feedback: This novel feedback loop acts as a regularizer. Reconstructed features are fed into a regressor that maps them back to the original embeddings, forcing the decoder to produce higher-quality, more representative samples.
Key Insights: Coarse-to-Fine Refinement
A critical aspect of this model is the iterative refinement process. By "unfolding" the feedback loop, the model simulates a recurrent structure. This allows the system to generate "coarse" samples in early iterations and "fine" samples later, mimicking how humans might gradually recognize complex objects. This feedback mechanism ensures that if the regressor cannot find the original attributes in the reconstructed feature, the generator is penalized, maintaining high quality-control over the synthesized features.
Experimental Results
The model was evaluated on four standard ZSL benchmarks: SUN Attribute, CUB (birds), and AWA1/AWA2 (animals). The researchers tested across both Standard Splits (SS) and Proposed Splits (PS) to ensure rigorous evaluation.
- Standard ZSL: On the AWA2 dataset, the model achieved 81.2% accuracy (SS), outperforming previous methods like SAE (80.8%) and ALE (80.3%).
- Generalized ZSL (GZSL): The model truly excels in the GZSL setting, where the test set contains both seen and unseen classes. It achieved a Harmonic Mean (H) of 76.1% on AWA1 and 75.3% on AWA2, nearly doubling or tripling the performance of many baseline models like ALE (27.5%) or DEVISE (22.4%).
Visualizations using t-SNE confirm that the predicted features for unseen classes cluster much more cleanly than raw features, proving the model successfully learns the underlying distribution of classes it never encountered during training.
Conclusion and Future Work
While the model demonstrates significant gains, particularly in GZSL, challenges remain in fine-grained recognition where visual differences are extremely subtle. Future work may involve exploring natural language descriptions instead of fixed attribute vectors and potentially integrating Generative Adversarial Network (GAN) architectures to further enhance the realism of synthetic features for unseen classes.
Summary updated