A Simple Exponential Family Framework for Zero-Shot Learning

Vinay Kumar Verma, Piyush Rai

Abstract

We present a simple generative framework for learning to predict previously unseen classes, based on estimating class-attribute-gated class-conditional distributions. We model each class-conditional distribution as an exponential family distribution and the parameters of the distribution of each seen/unseen class are defined as functions of the respective observed class attributes. These functions can be learned using only the seen class data and can be used to predict the parameters of the class-conditional distribution of each unseen class. Unlike most existing methods for zero-shot learning that represent classes as fixed embeddings in some vector space, our generative model naturally represents each class as a probability distribution. It is simple to implement and also allows leveraging additional unlabeled data from unseen classes to improve the estimates of their class-conditional distributions using transductive/semi-supervised learning. Moreover, it extends seamlessly to few-shot learning by easily updating these distributions when provided with a small number of additional labelled examples from unseen classes. Through a comprehensive set of experiments on several benchmark data sets, we demonstrate the efficacy of our framework.

Introduction

Zero-Shot Learning (ZSL) aims to recognize objects from classes that were never seen during training—for instance, identifying a "Zebra" having only seen "Horses" and being told a Zebra is a "striped horse." Most existing ZSL methods rely on semantic embeddings, mapping images and class descriptions (attributes) into a shared vector space. To classify a new image, they find the "closest" class point.

However, representing an entire class as a single fixed point ignores intra-class variability; not every instance of a class looks identical. By ignoring the "spread" or distribution of a class, these models struggle with complex, fine-grained datasets where classes might overlap or have high variance. This paper introduces a generative framework that models each class as a probability distribution rather than a single point in space.

Methodology

The core of the proposed framework is based on the Exponential Family of distributions. The process follows a structured pipeline to bridge the gap between seen and unseen classes using attribute-based parameter prediction.

Overview of the proposed generative framework for Zero-Shot Learning, illustrating the flow from empirical parameter estimation to regression and final refinement.

1. Classes as Distributions

Instead of a point, each class $c$ is modeled as a conditional distribution $p(x|\theta_c)$. While the framework works for any exponential family, the authors focus on the Gaussian case, where $ heta_c$ consists of a mean ($\mu_c$) and a variance ($\sigma_c^2$).

2. Attribute Gating

Since images for unseen classes are unavailable during training, the model learns a "gating" function (a regression model) that maps a class's high-level attributes (e.g., "has stripes") to its distribution parameters. The authors evaluated both Linear Models and Nonlinear Models (Kernel Ridge Regression) for this mapping.

3. Transductive and Few-Shot Extensions

  • Transductive Learning: If unlabeled images from unseen classes are available, the model uses an Expectation-Maximization (EM) procedure to refine the predicted distributions, effectively clustering the new data around the predicted class centers.
  • Few-Shot Extension: If a few labeled examples become available, the model treats the predicted parameters as conjugate priors. This allows for a mathematically elegant update of the class distribution using standard Bayesian identities.

Experiments and Results

The framework was evaluated on three standard ZSL benchmarks: Animals with Attributes (AwA), CUB-200-2011 (CUB), and SUN Attribute (SUN).

  • Inductive ZSL: The model achieved an average improvement of 5.67% across all three datasets compared to previous state-of-the-art methods. On the difficult CUB bird dataset, it showed a 21.05% relative improvement over the best baseline using VGG-19 features.
  • Transductive ZSL: By leveraging unlabeled test data, the model's accuracy on AwA increased from approximately 80% to over 94%, demonstrating its ability to overcome domain shift.
  • Few-Shot Learning: Even with just two labeled examples per unseen class, the model significantly outperformed standard multi-class SVMs, proving that the knowledge transferred from seen classes provides a superior starting point compared to learning from scratch.

Discussion and Conclusion

The primary insight of this work is that knowing the shape of a class (its variance) is as critical as knowing its center (its mean). This approach naturally accounts for class diversity and provides a unified path to transductive and few-shot learning. Unlike complex deep embedding architectures, the core of this model reduces to solving a ridge regression problem, making it computationally efficient and easy to implement.

While the model excels in standard ZSL, Generalized Zero-Shot Learning (GZSL)—where the test set contains both seen and unseen classes—remains a challenge due to the inherent bias toward seen classes. Future work may involve moving toward a fully Bayesian approach to better handle uncertainty in the attribute-to-parameter mapping.

Summary updated

Abstract

We present a simple generative framework for learning to predict previously unseen classes, based on estimating class-attribute-gated class-conditional distributions. We model each class-conditional distribution as an exponential family distribution and the parameters of the distribution of each seen/unseen class are defined as functions of the respective observed class attributes. These functions can be learned using only the seen class data and can be used to predict the parameters of the class-conditional distribution of each unseen class. Unlike most existing methods for zero-shot learning that represent classes as fixed embeddings in some vector space, our generative model naturally represents each class as a probability distribution. It is simple to implement and also allows leveraging additional unlabeled data from unseen classes to improve the estimates of their class-conditional distributions using transductive/semi-supervised learning. Moreover, it extends seamlessly to few-shot learning by easily updating these distributions when provided with a small number of additional labelled examples from unseen classes. Through a comprehensive set of experiments on several benchmark data sets, we demonstrate the efficacy of our framework.
BibTeX
@inproceedings{verma2017simple,
  author = {Vinay Kumar Verma and Piyush Rai},
  title = {{A Simple Exponential Family Framework for Zero-Shot Learning}},
  booktitle = {Ecml/pkdd},
  year = {2017},
  doi = {10.1007/978-3-319-71246-8\_48},
}
Figure