Learning to Separate Domains in Generalized Zero-Shot and Open Set Learning: a probabilistic perspective

Hanze Dong, Yanwei Fu, Leonid Sigal, Sung Ju Hwang, Yu-Gang Jiang, Xiangyang Xue

Abstract

Generalized Zero-Shot Learning (G-ZSL) aims to classify instances from both seen (source) and unseen (target) classes. A primary challenge in G-ZSL is the inherent prediction bias toward source classes. We propose a novel space decomposition method that partitions the feature space into Source, Target, and Uncertain spaces. By utilizing statistical tools such as bootstrapping and the Kolmogorov-Smirnov (K-S) test, we estimate and fine-tune the decision boundaries between these domains. Our approach achieves state-of-the-art results across multiple G-ZSL benchmarks.

Introduction

In traditional Zero-Shot Learning (ZSL), models are tested only on unseen classes. However, Generalized ZSL (G-ZSL) presents a more realistic scenario where the test set contains both seen and unseen classes. The fundamental difficulty is "prediction bias": since models are trained only on seen data, they tend to misclassify unseen instances as seen ones. Existing solutions often rely on fixed thresholds or treat unseen classes as outliers, which lack adaptability to varying data distributions.

Methodology: Compositional Space Decomposition

We introduce a framework that decomposes the data into three distinct compositional spaces to mitigate bias:

  1. Source Space: Contains instances predicted with high confidence as belonging to seen classes.
  2. Target Space: Contains instances likely belonging to unseen classes.
  3. Uncertain Space: A buffer zone for instances where the model's prediction is ambiguous.
Overview of the proposed pipeline and the visual/semantic embedding space decomposition.

To manage these spaces, we employ two key statistical techniques:

  • Bootstrapping: We use random sampling with replacement to estimate a robust, distribution-aware initial boundary between source and target classes, rather than relying on a hard-coded threshold.
  • Kolmogorov-Smirnov (K-S) Test: This parameter-free test compares the confidence score distribution of test instances against training data. If a significant difference is detected, the instance is moved to the Uncertain Space for further refinement.

In the Uncertain Space, the model performs a joint recognition task, considering both the top source class and potential target classes, effectively fine-tuning the decision boundary dynamically during inference.

Experimental Results

The proposed method was evaluated on standard benchmarks: Animals with Attributes (AwA), Caltech-UCSD Birds (CUB), Attribute Pascal and Yahoo (aPY), and large-scale ImageNet.

Performance comparison of the proposed method against state-of-the-art ZSL and G-ZSL models.

As shown in the results, our method significantly outperforms existing G-ZSL models, particularly in the Harmonic Mean ($H$), which balances accuracy for seen ($Acc_{S \to T}$) and unseen ($Acc_{U \to T}$) classes. For instance, on the AwA dataset, we achieved an $H$-score of 76.6%, a substantial improvement over the previous best of 64.2%. On the large-scale ImageNet dataset, our model maintained a competitive $H$-score of 10.3%, whereas many traditional models failed due to extreme source-class bias.

Conclusion

By introducing an "Uncertain Space" and leveraging distribution-based statistical tests, our method provides a robust solution to the prediction bias in G-ZSL. The parameter-free nature of the K-S test allows for seamless adaptation across different datasets. Future work will focus on integrating this decomposition directly into the training phase of generative models to further enhance boundary estimation.

Summary updated

Abstract

This paper studies the problem of Generalized Zero-shot Learning (G-ZSL), whose goal is to classify instances belonging to both seen and unseen classes at the test time. We propose a novel space decomposition method to solve G-ZSL. Some previous models with space decomposition operations only calibrate the confident prediction of source classes (W-SVM [46]) or take target-class instances as outliers [49]. In contrast, we propose to directly estimate and fine-tune the decision boundary between the source and the target classes. Specifically, we put forward a framework that enables to learn compositional spaces by splitting the instances into Source, Target, and Uncertain spaces and perform recognition in each space, where the uncertain space contains instances whose labels cannot be confidently predicted. We use two statistical tools, namely, bootstrapping and Kolmogorov-Smirnov (K-S) Test, to learn the compositional spaces for G-ZSL. We validate our method extensively on multiple G-ZSL benchmarks, on which it achieves state-of-the-art performances.
BibTeX
@misc{dong2018learning,
  author = {Hanze Dong and Yanwei Fu and Leonid Sigal and Sung Ju Hwang and Yu-Gang Jiang and Xiangyang Xue},
  title = {{Learning to Separate Domains in Generalized Zero-Shot and Open Set Learning: a probabilistic perspective}},
  year = {2018},
  eprint = {1810.07368},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1810.07368},
}
Figure