Learning the Compositional Spaces for Generalized Zero-shot Learning

Hanze Dong, Yanwei Fu, Sung Ju Hwang, Leonid Sigal, Xiangyang Xue

Abstract

This paper studies the problem of Generalized Zero-shot Learning (G-ZSL), whose goal is to classify instances belonging to both seen and unseen classes at the test time. We propose a novel space decomposition method to solve G-ZSL. Specifically, we put forward a framework that enables to learn compositional spaces by splitting the instances into Source, Target, and Uncertain spaces and perform recognition in each space. We use two statistical tools, namely, bootstrapping and Kolmogorov-Smirnov (K-S) Test, to learn the compositional spaces for G-ZSL. We validate our method extensively on multiple G-ZSL benchmarks, on which it achieves state-of-the-art performances.

Introduction

In standard Zero-Shot Learning (ZSL), models recognize "unseen" target classes by learning from "seen" source classes via shared semantic attributes. Generalized ZSL (G-ZSL) presents a more realistic challenge where the test set contains a mixture of both seen and unseen classes. The primary obstacle in G-ZSL is the inherent bias toward seen classes; since models are trained only on source data, they tend to overfit, incorrectly labeling unseen target instances as seen source instances at test time.

While previous methods attempt to mitigate this through confidence score calibration or data synthesis, this work proposes a "space decomposition" strategy. Instead of simple score adjustment, the framework explicitly models the decision boundaries between different data "spaces" to reduce bias and improve classification accuracy for both seen and unseen categories.

Methodology: Space Decomposition

The core of the proposed approach is the partitioning of the instance space into three distinct compositional regions:

  1. Source Space: Contains instances that the model identifies with high confidence as belonging to a known (seen) class.
  2. Target Space: Contains instances that clearly do not fit known classes, effectively acting as outliers relative to the source distribution.
  3. Uncertain Space: A "buffer zone" for instances that are ambiguous or where the classifier's performance is statistically questionable.
Visual representation of the Source, Uncertain, and Target spaces within the embedding space, showing how probability distributions are adjusted.

Statistical Boundary Estimation

To define these spaces without relying on arbitrary heuristics, the authors employ two statistical techniques:

  • Bootstrapping: This method is used on training confidence scores to estimate a robust, data-driven initial boundary for the Source space.
  • Kolmogorov-Smirnov (K-S) Test: This non-parametric test compares the distribution of confidence scores for test instances against the training distribution. If a significant mismatch is detected, instances are moved to the Uncertain Space, acknowledging the model's potential unreliability for those samples.

Recognition strategies vary by space: standard supervised classification is used in the Source space, while a mapping to semantic attribute space (nearest class prototype) is utilized for the Target and Uncertain spaces.

Experimental Results

The framework was evaluated on four major benchmarks: Animals with Attributes (AwA), CUB (birds), aPY (objects), and a large-scale ImageNet test. Performance is measured using the Harmonic Mean ($H$) of seen and unseen class accuracies.

Comparison of the proposed method against various ZSL and G-ZSL models on AwA, CUB, and aPY datasets, highlighting superior Harmonic Mean (H) scores.

As shown in the results table, the proposed method (Ours SVM/A-G) achieves significant improvements over existing state-of-the-art models. On the AwA dataset, it reached an $H$ score of 76.6%, a substantial jump from previous benchmarks. Notably, on the large-scale ImageNet benchmark—where many models fail to recognize unseen classes entirely—this method achieved an $H$ score of 10.3%, demonstrating its scalability to thousands of classes.

Conclusion

By introducing an "Uncertain" safety net and using rigorous statistical tests to define space boundaries, this decomposition framework effectively addresses the class bias problem in G-ZSL. While the method's performance depends on the quality of the initial supervised classifier, its ability to handle distributional mismatches makes it a robust solution for open-world recognition tasks. Future work may explore integrating generative models to further refine the Uncertain space.

Summary updated

Abstract

This paper studies the problem of Generalized Zero-shot Learning (G-ZSL), whose goal is to classify instances belonging to both seen and unseen classes at the test time. We propose a novel space decomposition method to solve G-ZSL. Some previous models with space decomposition operations only calibrate the confident prediction of source classes (W-SVM [46]) or take target-class instances as outliers [49]. In contrast, we propose to directly estimate and fine-tune the decision boundary between the source and the target classes. Specifically, we put forward a framework that enables to learn compositional spaces by splitting the instances into Source, Target, and Uncertain spaces and perform recognition in each space, where the uncertain space contains instances whose labels cannot be confidently predicted. We use two statistical tools, namely, bootstrapping and Kolmogorov-Smirnov (K-S) Test, to learn the compositional spaces for G-ZSL. We validate our method extensively on multiple G-ZSL benchmarks, on which it achieves state-of-the-art performances.
BibTeX
@misc{dong2018learning,
  author = {Hanze Dong and Yanwei Fu and Sung Ju Hwang and Leonid Sigal and Xiangyang Xue},
  title = {{Learning the Compositional Spaces for Generalized Zero-shot Learning}},
  year = {2018},
  eprint = {1810.07368},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1810.07368},
}
Figure