From Anchor Generation to Distribution Alignment: Learning a Discriminative Embedding Space for Zero-Shot Recognition

Fuzhen Li, Zhenfeng Zhu, Xingxing Zhang, Jian Cheng, Yao ZhaoarXiv 2020 · arXiv

Abstract

Zero-Shot Recognition (ZSR) faces significant challenges due to the irregular distribution of semantic attribute spaces and the misalignment between visual features and category templates. We present the Discriminative Anchor Generation and Distribution Alignment (DAGDA) framework, which addresses these issues by transforming raw attribute spaces into structured "anchor spaces." DAGDA utilizes a Truncated Diffusion Graph Convolutional Network (GCN) to model class-attribute interactions and a novel relation regularization technique to align visual distributions with semantic anchors. Our approach achieves state-of-the-art performance on major benchmarks, including AWA2 and CUB, demonstrating superior discriminative power in both conventional and generalized ZSR settings.

Introduction

Zero-Shot Learning (ZSL) aims to recognize objects from categories not seen during training by leveraging shared semantic attributes (e.g., "has wings," "is furry"). Typically, models project visual features into a semantic space where seen and unseen classes coexist. However, raw attribute templates often suffer from irregular distributions, where classes cluster too closely or the mapping from pixels to attributes is noisy, leading to poor generalization.

Overview of the Zero-Shot Recognition challenge, showing the mapping between visual space, attribute vectors, and the semantic attribute space.

The DAGDA framework tackles these challenges by creating a more structured, discriminative embedding space. By generating optimized "anchors" for each category and ensuring tight alignment between visual samples and these anchors, the model improves class separation and recognition accuracy.

Methodology

The DAGDA framework consists of two primary stages:

1. Discriminative Anchor Generation

Instead of using raw, static attribute templates as targets, DAGDA treats classes and attributes as nodes in a bipartite graph. A Truncated Diffusion Graph Convolutional Network (GCN) is employed to learn low-dimensional "anchors" for each class.

  • Diffusion Mechanism: Unlike standard GCNs that only consider immediate neighbors, diffusion captures higher-order interactions between classes and attributes, rectifying the semantic space to be more discriminative.
  • Truncation: To avoid "over-smoothing"—where node representations become indistinguishable—the diffusion process is limited to a specific number of steps, preserving the local manifold structure.

2. Distribution Alignment

Once anchors are established, the model learns a projection from the visual feature space to the anchor space using a symmetrical auto-encoder. A key innovation is Relation Regularization, which ensures that the local distribution of visual samples around a class anchor mirrors the semantic relationships defined by the attributes. This alignment is crucial for maintaining the fine-grained structure necessary for distinguishing similar classes.

Experimental Evaluation

The model was evaluated on four standard ZSL benchmarks: AWA2 (Animals with Attributes 2), CUB (Caltech UCSD Birds 200), SUN (Scene Recognition), and aPY (Attribute Pascal and Yahoo). Visual features were extracted using a pre-trained ResNet-101.

Key Results

DAGDA demonstrated significant improvements over existing state-of-the-art methods:

  • Conventional ZSL: Achieved 73.0% accuracy on AWA2 and 64.1% on CUB, outperforming baselines like ALE and SYNC.
  • Generalized ZSL: On the fine-grained CUB dataset, it reached an H-measure of 35.2%, surpassing models like SAE and PSRZSL.
  • Ablation Analysis: Removing either the GCN-based anchor generation or the relation regularization resulted in a significant performance drop, confirming that both structural rectification and distribution alignment are essential.

Discussion and Conclusion

The core strength of DAGDA lies in its graph-based semantic rectification and the use of optimized anchors rather than static templates. By modeling class-attribute interactions, the system discovers latent structures that raw vectors miss.

While highly effective, the model shows some sensitivity to graph scale, with slightly lower performance on smaller datasets like aPY (32 classes). Future work may explore the use of implicit side information, such as raw text descriptions, to reduce dependency on explicit attribute labels and improve flexibility across diverse domains.

Summary updated

Abstract

In zero-shot learning (ZSL), the samples to be classified are usually projected into side information templates such as attributes. However, the irregular distribution of templates makes classification results confused. To alleviate this issue, we propose a novel framework called Discriminative Anchor Generation and Distribution Alignment Model (DAGDA). Firstly, in order to rectify the distribution of original templates, a diffusion based graph convolutional network, which can explicitly model the interaction between class and side information, is proposed to produce discriminative anchors. Secondly, to further align the samples with the corresponding anchors in anchor space, which aims to refine the distribution in a fine-grained manner, we introduce a semantic relation regularization in anchor space. Following the way of inductive learning, our approach outperforms some existing state-of-the-art methods, on several benchmark datasets, for both conventional as well as generalized ZSL setting. Meanwhile, the ablation experiments strongly demonstrate the effectiveness of each component.
BibTeX
@misc{li2020from,
  author = {Fuzhen Li and Zhenfeng Zhu and Xingxing Zhang and Jian Cheng and Yao Zhao},
  title = {{From Anchor Generation to Distribution Alignment: Learning a Discriminative Embedding Space for Zero-Shot Recognition}},
  year = {2020},
  eprint = {2002.03554},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2002.03554},
}
Figure