Abstract
Generalized Zero-Shot Learning (GZSL) faces a significant challenge in seen-class bias, where models overfit to seen distributions and struggle to generalize to unseen categories. We present D³GZSL, a novel framework that reformulates GZSL as an Out-of-Distribution (OOD) detection task. By leveraging a Teacher-Student distillation architecture, D³GZSL aligns the distributions of real seen data and synthesized unseen features. Our approach includes In-Distribution Dual-Space Distillation ($ID^2SD$) and Out-of-Distribution Batch Distillation ($O^2DBD$) to sharpen decision boundaries. Experimental results across four major benchmarks demonstrate that D³GZSL consistently improves the performance of various generative baselines, setting new state-of-the-art results.
Introduction
In computer vision, Zero-Shot Learning (ZSL) aims to recognize objects from classes not seen during training by leveraging semantic descriptions. Generalized ZSL (GZSL) extends this to a more realistic scenario where the test set contains both seen and unseen classes. The primary hurdle in GZSL is the inherent bias toward seen classes; because models are trained on real images of seen classes but only synthetic features of unseen classes, they often misclassify unseen samples as belonging to the seen distribution.
Current generative methods attempt to mitigate this by synthesizing features for unseen classes. However, these methods often fail to capture the nuanced boundaries between "In-Distribution" (seen) and "Out-of-Distribution" (unseen) data. D³GZSL addresses this by treating the GZSL problem through the lens of OOD detection, using knowledge distillation to bridge the gap between real and synthetic data distributions.
Proposed Method: D³GZSL
The D³GZSL framework is a "plug-and-play" architecture that can be integrated with various generative models, such as GANs, VAEs, or Diffusion models. It consists of three primary components designed to refine the feature space and reduce classification bias.
1. Feature Generation (FG)
This stage utilizes a baseline generative model to synthesize features for unseen classes based on their semantic attributes. These synthetic features serve as the initial representation of the unseen distribution.
2. In-Distribution Dual-Space Distillation ($ID^2SD$)
To ensure the model retains high performance on seen classes while learning unseen ones, $ID^2SD$ employs a Teacher-Student architecture. A teacher model, pre-trained on real seen data, guides a student model by aligning them in two critical areas:
- Embedding Space: Capturing category correlations.
- Label Space: Aligning logit outputs to mimic the teacher’s confidence levels.
3. Out-of-Distribution Batch Distillation ($O^2DBD$)
This component treats synthesized unseen data as OOD samples. It maps softmax probabilities into a low-dimensional OOD representation space. By modeling correlations within a batch, the framework learns "shared structures" between seen and unseen categories, allowing for more effective discrimination at the decision boundary.
Experimental Results
The authors evaluated D³GZSL on four standard benchmarks: AWA1, AWA2, CUB (fine-grained birds), and FLO (fine-grained flowers). The framework was tested as an enhancement to several state-of-the-art baselines, including f-CLSWGAN, TF-VAEGAN, and CE-GZSL.
As shown in the results table, D³GZSL consistently boosted performance across all metrics. Notably, on the CUB dataset, applying D³GZSL to the f-CLSWGAN baseline increased the Harmonic Mean ($H$-score) from 49.7% to 56.5%. The most significant gains were observed in the accuracy of unseen classes ($U$), demonstrating the framework's effectiveness in reducing seen-class bias. For instance, on AWA1, the CE-GZSL baseline achieved a top-tier $H$-score of 70.5% when augmented with D³GZSL.
Conclusion
D³GZSL represents a significant shift in GZSL methodology by reframing the task as an OOD detection problem. By utilizing knowledge distillation to align real and synthetic distributions, the framework effectively mitigates seen-class bias. While the method's performance is partially dependent on the quality of the teacher model and introduces some computational complexity, its versatility across different generative architectures makes it a powerful tool for advancing zero-shot recognition capabilities.
Summary updated