Introduction
Zero-Shot Learning (ZSL) aims to recognize objects from classes unseen during training by leveraging semantic descriptions, such as attributes. While normalization techniques like BatchNorm are standard in deep learning, their application in ZSL has often relied on heuristics. This paper provides a theoretical foundation for these "tricks" and introduces Class Normalization (CN), a technique that enables simple neural networks to outperform complex state-of-the-art models.
The authors identify three critical challenges in ZSL:
- Theoretical Gap: Widely used practices like attribute normalization and scaled cosine similarity lack rigorous explanation.
- Deep Model Instability: Heuristics that stabilize linear models often fail in deeper architectures.
- Irregular Loss Surfaces: ZSL optimization landscapes are significantly more "bumpy" than those of standard supervised classifiers.
Methodology
The researchers address these issues through variance propagation theory and architectural innovation.
Variance Analysis and Scaling
The paper demonstrates that common ZSL tricks are essentially mechanisms for preserving signal variance. They derive a formula for the optimal scaling factor ($\gamma$) in cosine similarity based on feature space dimensionality, ensuring output logit variance remains near 1.0 to prevent gradient vanishing.
Class Normalization (CN)
The primary contribution, Class Normalization, standardizes attribute embeddings class-wise before the final projection. By ensuring constant-norm inputs to the last layer and pairing this with variance-preserving weight initialization, the model achieves significantly better stability.
Continual ZSL (CZSL)
The authors also propose a framework for Continual ZSL, where models must learn a sequence of tasks without forgetting previously learned unseen classes.
Key Insights: Smoothing the Optimization Landscape
A major finding is that ZSL models receive $K$ times more gradient signals per example than standard classifiers (where $K$ is the number of classes), leading to highly irregular loss surfaces. Class Normalization acts as a smoother, bringing the optimization landscape closer to that of traditional supervised learning.
Experimental Results
The proposed "vanilla" 3-layer MLP with Class Normalization was evaluated on SUN, CUB, AwA1, and AwA2 benchmarks.
Performance and Efficiency
Despite its simplicity, the CN-enhanced MLP achieved state-of-the-art results on three of the four datasets. Most notably, it offers a massive efficiency gain: training takes approximately 30 seconds, compared to the 1.5 to 3 hours required by complex generative models like GANs or VAEs.
Continual Learning Impact
In the CZSL benchmark, Class Normalization improved the performance of existing algorithms (e.g., EWC, A-GEM) by up to 100% in certain metrics, highlighting its importance for lifelong learning in semantic spaces.
Conclusion
This work demonstrates that proper normalization is often more critical than model complexity in ZSL. While simple MLPs with CN show remarkable performance, the gap between ZSL and fully supervised learning remains significant. Future research may focus on more sophisticated attribute-to-visual mappings that maintain these normalization benefits while capturing deeper semantic relationships.
Summary updated