Introduction
Zero-Shot Learning (ZSL) aims to recognize object classes that were never seen during training by leveraging auxiliary information, such as attribute vectors (e.g., "has stripes," "is yellow"). Despite its potential, ZSL faces two primary challenges: Domain Shift and High-Bias. Domain shift occurs when the mapping from image features to attributes for seen classes fails to generalize to unseen classes due to differing high-dimensional distributions. High-bias is prevalent in Generalized ZSL (GZSL), where models naturally favor seen classes because they lack real training data for unseen ones.
Traditional methods often attempt point-by-point matching between samples and attributes, which is highly sensitive to noise. This paper proposes a shift from matching individual points to matching entire distributions using an Optimal Transport (OT) framework.
Methodology
The proposed OT-ZSL framework utilizes a generative approach to bridge the gap between seen and unseen classes.
Conditional Feature Generator
Instead of mapping images to attributes, the model employs a generator $g$ that synthesizes "fake" image features from attribute vectors and noise. This allows the model to "see" representations of unseen classes during training.
Optimal Transport (OT) Loss
To ensure the synthetic features are realistic, the model minimizes the Wasserstein distance (OT distance) between the distribution of generated features and real features extracted from a pretrained CNN. By solving the OT problem in its primal form, the authors avoid the stability issues common in Wasserstein GANs (WGANs), such as the need for strict 1-Lipschitz constraints.
IPOT Algorithm and Regularization
The Inexact Proximal point method for Optimal Transport (IPOT) is used to solve the OT problem efficiently and robustly. To maintain discriminative power and prevent "mode collapse," an attribute predictor $f$ is trained alongside the generator, forcing synthetic features to map accurately back to their original class attributes.
Datasets and Evaluation
The model was evaluated on several standard benchmarks, including AwA1 & AwA2 (Animals with Attributes), CUB (Caltech-UCSD Birds-200), SUN (Scene Recognition), and a large-scale test on ImageNet involving 1,000 seen and 360 unseen classes.
Key Results and Insights
OT-ZSL achieved state-of-the-art performance across multiple settings. In Generalized ZSL, it reached a Harmonic Mean ($H$) score of 69.6% on AwA1, significantly outperforming previous generative methods like f-CLSWGAN (59.6%). For standard ZSL, it achieved 74.3% accuracy on AwA1 and 68.1% on CUB.
The effectiveness of the generator is visually confirmed via t-SNE, where synthetic features for unseen classes (e.g., "horse," "sheep") cluster tightly with real, withheld features. By synthesizing data for unseen classes, the final softmax classifier is trained on a balanced dataset, drastically reducing bias toward seen classes. Furthermore, using cosine distance within the OT framework helped mitigate the "hubness" problem common in high-dimensional embeddings.
Conclusion
By framing Zero-Shot Learning as a distribution matching problem via Optimal Transport, this work effectively addresses domain shift and class bias. While the model can still struggle with "semantic confusion" between highly similar classes (e.g., blue whales vs. dolphins), the OT-ZSL framework provides a robust and scalable solution for recognizing the unseen.
Summary updated