Standard supervised classification models operate under a strict assumption: all categories encountered during testing must be present during training. When a model is presented with an instance from an entirely new category, it fails because it lacks a trained classification head for that label.
Zero-Shot Learning (ZSL) and Generalized Zero-Shot Learning (GZSL) break this constraint by decoupling classification from explicit label IDs. Instead of mapping data directly to discrete classes, ZSL frameworks map data into an intermediate, shared semantic space—such as text descriptions, attribute vectors, or knowledge graphs. Because novel categories can also be described in this semantic space, the model can recognize unseen classes by finding the closest match between the extracted data features and the semantic definitions of the target classes.
However, executing this alignment effectively requires overcoming three core challenges: seen-class bias (the tendency to over-fit to trained categories), domain discrepancy (the gap between how attributes manifest visually versus textually), and spurious correlations (latching onto non-transferable background features).
Mapping Spaces Bidirectionally and Jointly
The earliest formulations of ZSL relied on a unidirectional projection, typically mapping visual features directly into a semantic attribute space. This approach often suffers from the "hubness problem," where high-dimensional projections cause a small subset of target vectors to become the nearest neighbors for nearly all test samples.
To counter this, researchers shifted toward bidirectional and joint embedding spaces. Huadong Sun et al. demonstrate in Embedded Zero-Shot Image Classification Based on Bidirectional Feature Mapping that learning a dual-mapping framework—simultaneously projecting visual features to semantic spaces and semantic attributes back to visual spaces—captures intricate, reciprocal relationships that unidirectional models miss. This closed-loop alignment significantly mitigates seen-class bias.
Unidirectional: Visual Features ───────────────> Semantic Space (Hubness Risk)
Bidirectional: Visual Features <==============> Semantic Space (Closed-Loop Alignment)
Rather than projecting one space into another, an alternative strategy forces both modalities into a shared latent space. Mohammad Rostami et al. explore this in Zero-shot image classification using coupled dictionary embedding, using coupled dictionary learning to enforce a shared sparse representation for both visual and semantic features.
When evaluating these mapping paradigms, a comprehensive benchmark study by Yongqin Xian et al., Zero-Shot Learning - A Comprehensive Evaluation of the Good, the Bad and the Ugly, revealed that direct compatibility learning methods—which optimize a compatibility score matrix between visual and semantic features using max-margin losses—consistently outperform disjoint attribute classifiers.
Synthesizing Unseen Data via Relational Knowledge
When real visual samples for novel classes are entirely unavailable, a powerful alternative is to transform ZSL into a standard supervised problem by synthesizing the missing visual features.
To ensure that generated features are realistic and structurally sound, models must exploit the underlying structure of the semantic space. Donghui Wang et al. address this in Relational Knowledge Transfer for Zero-Shot Learning by measuring the geometric and structural relationships among classes within the semantic attribute space. They use these relationships to guide the generation of virtual data for unseen categories, effectively transferring knowledge from well-sampled seen domains to empty unseen domains.
[Seen Class Semantics] ───(Extract Relations)───> [Unseen Class Semantics]
│ │
▼ ▼
[Real Visual Features] ───(Guide Generation)────> [Synthesized Visual Features]
To handle highly descriptive, unformatted semantic inputs, Zhi Chen et al. introduce CANZSL in Canzsl: Cycle-Consistent Adversarial Networks for Zero-Shot Learning from Natural Language. Instead of relying on clean, hand-engineered attribute vectors, their framework uses cycle-consistent adversarial networks to synthesize realistic visual features directly from noisy, free-form natural language descriptions. The cycle-consistency constraint ensures that synthesized visual features retain enough distinct information to be mapped back to their original text descriptions, preventing mode collapse during generation.
Fine-Grained Alignment in Vision-Language Models
The emergence of massive, pre-trained Vision-Language Models (VLMs) like CLIP has shifted the ZSL landscape. Instead of training alignment layers from scratch, modern techniques adapt these foundation models to downstream tasks. However, standard VLMs optimize for global image-text similarity, which often glosses over fine-grained details or suffers from textual redundancy.
To resolve this, Yuhao Sun et al. propose BiFTA in Let's Roll a BiFTA: Bi-refinement for Fine-grained Text-visual Alignment in Vision-Language Models. BiFTA introduces a dual refinement mechanism that prunes redundant image patches and filters out uninformative words in textual descriptions simultaneously, forcing the VLM to align highly specific visual regions with precise descriptive phrases.
Taking local alignment a step further, the authors of Interpretable Zero-Shot Learning with Locally-Aligned Vision-Language Model introduce LaZSL. Rather than relying on a single global dot-product, LaZSL uses Optimal Transport to explicitly align local visual patches with individual semantic attributes. This localized matching provides a clear mathematical explanation of why an image matches an unseen class, offering built-in interpretability without requiring model fine-tuning.
Global VLM Alignment: [ Entire Image ] <─────── Similarity Score ───────> [ "A spotted bird with a yellow beak" ]
Local (LaZSL) Alignment: [ Patch A (Beak) ] <─── Optimal Transport Alignment ───> [ "yellow beak" ]
[ Patch B (Body) ] <─── Optimal Transport Alignment ───> [ "spotted" ]
When applying VLMs to structured domains like document layouts, standard text prompts fall short. Sankalp Sinha et al. address this in CICA: Content-Injected Contrastive Alignment for Zero-Shot Document Image Classification by injecting explicit textual content extracted from document images directly into the CLIP embedding space via a coupled-contrastive loss, ensuring the model balances both visual layout and textual content.
De-biasing and Causal Inference
In Generalized Zero-Shot Learning (GZSL), the test set contains a mixture of both seen and unseen classes. Because the model's feature extractor was optimized exclusively on seen classes, it exhibits an overwhelming bias toward predicting seen labels at test time.
Debasmit Das et al. demonstrate in Zero-shot Image Recognition Using Relational Matching, Adaptation and Calibration that this issue must be attacked from multiple angles. By combining relational matching (to preserve class structures), test-time domain adaptation, and explicit score calibration, they prevent seen classes from dominating the prediction scores.
To fix this bias at the root, models must avoid learning non-transferable shortcuts. Shuhuang Chen et al. identify this as a causal confounding problem in Causal Visual-semantic Correlation for Zero-shot Learning. They introduce a Causal Visual-semantic Correlation (CVsC) framework that uses causal inference and a Conditional Correlation Loss to break spurious correlations (such as associating a "duck" attribute exclusively with a "water" background), ensuring the model focuses on true, transferable semantic attributes.
Spurious Correlation (Confounded): [ Visual Features (Water) ] ───> [ Semantic Attribute (Duck) ]
Causal Inference (De-confounded): [ Visual Features (Body) ] ───> [ Semantic Attribute (Duck) ]
On a structural level, the authors of Hierarchical Divide-and-Conquer Grouping for Classification Adaptation of Pre-Trained Models introduce the HDG framework. HDG systematically partitions the label space into distinct base (seen) and novel (unseen) subspaces, and then refines these groups hierarchically to eliminate intra-domain confusion before final classification occurs.
Beyond Standard Images: Compositional and Structured Domains
The core philosophy of zero-shot alignment—mapping raw data structures to structured semantic priors—extends far beyond standard image classification.
Compositional Zero-Shot Learning
In Compositional ZSL, the goal is to recognize novel combinations of known primitives, such as an unseen state applied to a seen object. Yun Li et al. tackle this in Simple Primitives With Feasibility- and Contextuality-Dependence for Open-World Compositional Zero-Shot Learning by explicitly modeling the feasibility (whether a state and object can logically coexist) and contextuality (how an object's appearance changes based on its state) dependencies between visual primitives.
Structured and Graph-Based Domains
When dealing with highly structured visual data like Chinese characters, Yang Hong et al. introduce FT-CLIP in Improving Chinese character representation with formation tree. Instead of treating characters as flat images, they encode their hierarchical radical structures ("formation trees") and align them with visual representations, enabling the zero-shot recognition of entirely unseen characters.
In the biological and chemical sciences, zero-shot alignment operates over graph structures:
- Single-Cell Analysis: Chen Fang et al. develop CGCompass in Cell-GraphCompass: modeling single cells with graph structure foundation model, a graph neural network foundation model that incorporates biological prior knowledge into a graph structure to perform zero-shot cell type annotation.
- Drug Discovery: Qing Ye et al. introduce KGDRP in A Knowledge‐Guided Graph Learning Approach Bridging Phenotype‐ and Target‐Based Drug Discovery, unifying phenotype- and target-based data within a heterogeneous graph to enable zero-shot drug repurposing for emerging diseases.
Dense Prediction and Downstream Tasks
The paradigm also adapts to dense regression and localization tasks. For instance, Don Yasiru Lakshan Ranasinghe et al. present ReGe-Count in Retrieval-based Zero-shot Crowd Counting, which uses a two-stage retrieval process to pull knowledge from simulated datasets, allowing a VLM to count crowds in real-world images without manual annotations. In object detection, Berkan Demirel et al. implement Zero-Shot Object Detection by Hybrid Region Embedding, combining bounding-box region proposals with hybrid semantic embeddings within a YOLO framework to localize and classify unseen object categories simultaneously.
Key Foundational Work
Transductive Multi-View Zero-Shot Learning. (2015) — contributed 1 concept elements to this direction.
NuActiv: recognizing unseen new activities using semantic attribute-based learning (2013) — contributed 1 concept elements to this direction.