Multi-label Few-shot Learning with Semantic Inference (Student Abstract)

Zhen Wang, Yiqun Duan, Liu Liu, Dacheng Tao

Abstract

Multi-label Few-shot Learning (ML-FSL) presents unique challenges due to the exponential growth of the output space and the scarcity of labeled data. This paper introduces DESIRE-Net (DEep Semantic InfeREnce Network), a framework that leverages semantic word embeddings and a novel inference mechanism to exploit label correlations. By mapping features into a structured semantic space and utilizing prior knowledge from base labels, DESIRE-Net significantly outperforms traditional meta-learning approaches in data-sparse, high-dimensional environments.

1. Introduction

Standard few-shot learning typically addresses single-label classification. However, real-world data is often multi-label, where an instance (e.g., an image or document) belongs to multiple categories simultaneously. ML-FSL is particularly difficult for two primary reasons:

  1. Exponential Output Space: With $N$ possible labels, there are $2^N$ possible combinations, creating a massive, sparse search space.
  2. The Low-Data Trap: Limited examples per label make models highly susceptible to overfitting when navigating this complex output space.

This work proposes "semantic inference" to bridge this gap, teaching the model to use linguistic relationships between words to understand and predict novel categories.

2. The DESIRE-Net Framework

The DEep Semantic InfeREnce Network (DESIRE-Net) incorporates two key innovations built upon a gradient-based meta-learning paradigm:

  • Semantic-Aware Feature Extractor: Instead of traditional one-hot encoding, the model uses word embedding vectors (e.g., Word2Vec or GloVe). This maps features into a "semantic space" where conceptually similar labels (like "car" and "truck") are mathematically proximal. This structures the output space and acts as a regularizer, forcing the model to align features with human-like semantic concepts.
  • Semantic Inference Mechanism: This specialized layer leverages prior knowledge from "base" labels (seen during training) to help classify "novel" labels (seen only a few times). It uses a learnable correlation matrix ($W_I$) to map relationships between known and unknown categories, effectively predicting the presence of a new label based on its similarity to known ones.

3. Experimental Evaluation

The model was evaluated using the Delicious dataset, a multi-label benchmark consisting of 983 labels and over 16,000 examples. The authors partitioned these into 600 base labels for initial training, 175 validation labels, and 200 novel labels for the few-shot testing phase. Testing was conducted in 5-way and 10-way settings with 1-shot and 5-shot examples.

4. Results and Analysis

DESIRE-Net consistently outperformed established meta-learning baselines such as MAML, Reptile, and ATAML across all tested configurations.

Comparative performance of DESIRE-Net and baselines on the Delicious dataset.

As shown in the table above, DESIRE-Net achieved an AUC of 87.2% in the 5-way 1-shot setting, compared to MAML’s 75.9%. In the 10-way 5-shot setting, it reached 88.7%.

Ablation studies confirmed that every component of the architecture is vital. Removing the semantic inference mechanism ($\setminus\mathcal{I}$) or the semantic feature extractor ($\setminus\mathcal{F}$) led to significant performance drops of roughly 7-9%. Furthermore, the inclusion of $l_2$ normalization in the inference mechanism proved crucial for preventing feature magnitude from over-influencing results, enhancing model robustness.

5. Conclusion and Future Work

DESIRE-Net demonstrates that explicitly modeling label correlations through semantic embeddings is a powerful strategy for ML-FSL. While the results on the text-based Delicious dataset are promising, future work should explore the architecture's performance on complex image datasets like MS-COCO or NUS-WIDE. Additionally, investigating the computational overhead of the inference layer as the number of base labels scales, and exploring the joint fine-tuning of word embeddings for domain-specific tasks, remain important avenues for research.

Summary updated

Abstract

Few-shot learning can adapt the classification model to new labels with only a few labeled examples. Previous studies mainly focus on the scenario of a single category label per example but have not solved the more challenging multi-label scenario with exponential-sized output space and low-data effectively. In this paper, we propose a semantic-aware meta-learning model for multi-label few-shot learning. Our approach can learn and infer the semantic correlation between unseen labels and historical labels to quickly adapt multi-label tasks from only a few examples. Specifically, features can be mapped into the semantic embedding space via label word vectors to explore and exploit the label correlation, and thus cope with the challenge on the overwhelming size of the output space. Then a novel semantic inference mechanism is designed for leveraging prior knowledge learned from historical labels, which will produce good generalization performance on new labels to alleviate the low-data problem. Finally, extensive empirical results show that the proposed method significantly outperforms the existing state-of-the-art methods on the multi-label few-shot learning tasks.
BibTeX
@inproceedings{wang2021multi,
  author = {Zhen Wang and Yiqun Duan and Liu Liu and Dacheng Tao},
  title = {{Multi-label Few-shot Learning with Semantic Inference (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.17955},
}
Figure