A Simple Exponential Family Framework for Zero-Shot Learning

Vinay Kumar Verma, Piyush Rai

TL;DR

This paper introduces a simple yet powerful generative framework for Zero-Shot Learning (ZSL) that models each class as a probability distribution rather than a single point in space. By using an exponential family framework where distribution parameters are predicted from class attributes, the authors create a versatile model that excels at standard ZSL, leverages unlabeled data (transductive learning), and seamlessly transitions into few-shot learning.

What Problem Does This Paper Address?

Zero-Shot Learning (ZSL) aims to recognize objects from classes that were never seen during training (e.g., identifying a "Zebra" having only seen "Horses" and being told a Zebra is a "striped horse").

Most existing ZSL methods rely on semantic embeddings: they map images and class descriptions (attributes) into a shared vector space. To classify a new image, they find the "closest" class point. The authors identify a major flaw here: representing an entire class as a single fixed point ignores intra-class variability. Not every "Zebra" looks identical. By ignoring the "spread" or distribution of a class, these models struggle with complex, fine-grained datasets where classes might overlap or have high variance.

Methods

The authors propose a generative framework based on the Exponential Family of distributions.

  1. Classes as Distributions: Instead of a point, each class $c$ is modeled as a conditional distribution $p(x|\theta_c)$. While the framework works for any exponential family, they focus on the Gaussian case, where $\theta_c$ consists of a mean ($\mu_c$) and a variance ($\sigma_c^2$).
  2. Attribute Gating: Since we have no images for unseen classes, we can't calculate $\theta_c$ directly. Instead, the model learns a "gating" function (a regression model) that maps a class's high-level attributes (like "has stripes," "is black and white") to its distribution parameters.
  3. They test both Linear Models and Nonlinear Models (Kernel Ridge Regression) for this mapping.
  4. Transductive Learning: If unlabeled images from unseen classes are available, the model uses an Expectation-Maximization (EM) procedure to refine the predicted distributions, effectively "clustering" the new data around the predicted class centers.
  5. Few-Shot Extension: If a few labeled examples become available, the model treats the predicted parameters as conjugate priors. This allows for a mathematically elegant update of the class distribution using standard Bayesian identities.

Datasets

The framework was evaluated on three standard ZSL benchmarks:

  • Animals with Attributes (AwA): 30,475 images across 50 animal classes with 85-dimensional attribute vectors.
  • CUB-200-2011 (CUB): A fine-grained dataset of 11,788 images of 200 bird species.
  • SUN Attribute (SUN): 14,340 images across 717 scene categories.

Key Insights and Ideas

  • Distribution vs. Point: The "aha moment" is that knowing the shape of a class (its variance) is just as important as knowing its center (its mean). This naturally accounts for the fact that some classes are more diverse than others.
  • Simplicity is a Feature: Unlike complex deep embedding architectures, the core of this model reduces to solving a ridge regression problem, which has a closed-form solution. It’s computationally efficient and easy to implement.
  • Unified Framework: The generative nature provides a "free" path to transductive and few-shot learning. Because the model understands classes as distributions, adding a few samples or unlabeled data is just a matter of updating distribution statistics.

Key Results

  • Inductive ZSL: The model achieved an average improvement of 5.67% across all three datasets compared to the previous state-of-the-art. On the difficult CUB bird dataset, it showed a 21.05% relative improvement over the best baseline using VGG-19 features.
  • Transductive ZSL: By using unlabeled test data, the model's accuracy on AwA jumped from ~80% to over 94%, demonstrating its ability to overcome "domain shift" (when training and test distributions differ).
  • Few-Shot Learning: Even with just 2 labeled examples per unseen class, the model significantly outperformed standard multi-class SVMs, proving that the knowledge transferred from seen classes provides a much better starting point than learning from scratch.

Limitations and Future Work

The authors note that Generalized Zero-Shot Learning (GZSL)—where the test set contains both seen and unseen classes—remains a significant challenge. Most ZSL models, including this one, are heavily biased toward predicting "seen" classes because they have more data.

Future directions suggested include:

  • Jointly learning class attributes from external sources (like Wikipedia) alongside the image model.
  • Moving from point estimates of parameters to a fully Bayesian approach to better handle uncertainty in the attribute-to-parameter mapping.

Summary updated

Abstract

We present a simple generative framework for learning to predict previously unseen classes, based on estimating class-attribute-gated class-conditional distributions. We model each class-conditional distribution as an exponential family distribution and the parameters of the distribution of each seen/unseen class are defined as functions of the respective observed class attributes. These functions can be learned using only the seen class data and can be used to predict the parameters of the class-conditional distribution of each unseen class. Unlike most existing methods for zero-shot learning that represent classes as fixed embeddings in some vector space, our generative model naturally represents each class as a probability distribution. It is simple to implement and also allows leveraging additional unlabeled data from unseen classes to improve the estimates of their class-conditional distributions using transductive/semi-supervised learning. Moreover, it extends seamlessly to few-shot learning by easily updating these distributions when provided with a small number of additional labelled examples from unseen classes. Through a comprehensive set of experiments on several benchmark data sets, we demonstrate the efficacy of our framework.
BibTeX
@inproceedings{verma2017simple,
  author = {Vinay Kumar Verma and Piyush Rai},
  title = {{A Simple Exponential Family Framework for Zero-Shot Learning}},
  booktitle = {Ecml/pkdd},
  year = {2017},
  doi = {10.1007/978-3-319-71246-8\_48},
}

Loading related content...

Figure