Don't Miss the Labels: Label-semantic Augmented Meta-Learner for Few-Shot Text Classification

Qiaoyang Luo, Lingqiao Liu, Yuhao Lin, Wei ZhangACL 2021 · DOI

Introduction

Few-shot learning (FSL) aims to enable AI models to learn new concepts from a minimal number of examples, mirroring human cognitive abilities. In Natural Language Processing (NLP), meta-learning approaches like Prototypical Networks have become the standard. These models learn to map input text into a feature space where similar classes are clustered. However, a significant limitation persists: most existing models treat class labels as arbitrary integers, completely ignoring the rich semantic information inherent in class names.

For instance, distinguishing between "Can you help me pay my water bill?" and "Tell me the last day I can pay my gas bill" is challenging without the context of their respective categories, "Pay Bill" and "Bill Due." Humans naturally use label semantics to resolve such ambiguities, a capability this research seeks to replicate in AI.

The LaSAML Framework

The authors propose LaSAML (Label-semantic Augmented Meta-Learner), a framework designed for label-semantic guided feature extraction. The core innovation lies in its "plug-and-play" input augmentation strategy.

Instead of processing a sentence in isolation, LaSAML appends the ground-truth class name to the input: [CLS] sentence [SEP] class name [SEP]. This format leverages BERT’s pre-trained "Next Sentence Prediction" (NSP) mechanism, forcing the model to extract sentence features that are specifically relevant to the provided class name.

The LaSAML architecture showing the support and query processing paths through BERT to the classifier.

In a Prototypical Network (PN) setup, the "prototype" for a class is calculated by averaging the BERT embeddings of support samples augmented with their own class names. For query samples where the label is unknown, the framework can either process the sentence alone or compare it against embeddings generated by appending all possible class names to the query.

Experimental Setup

The effectiveness of LaSAML was evaluated across three diverse benchmarks:

  • HuffPost: News headlines across various categories.
  • Banking77: A fine-grained dataset with 77 banking-related intents.
  • Clinc150: A cross-domain dataset used to test generalization to entirely new domains (e.g., from "Travel" to "Kitchen").

Results and Discussion

The experimental results demonstrate that incorporating label semantics provides a massive performance boost, particularly in ultra-low-data scenarios.

Performance comparison of LaSAML-PN against various BERT-based baselines across three datasets.

Key findings include:

  • Significant 1-Shot Gains: On the HuffPost dataset, LaSAML-PN achieved 62.16% accuracy in 1-shot tasks, a ~16% improvement over standard BERT-based Prototypical Networks (46.11%).
  • Cross-Domain Robustness: On the Clinc150 dataset (15-way 1-shot), LaSAML-PN outperformed the nearest baseline by nearly 10%, suggesting that label semantics are crucial when encountering unseen domains.
  • Simplicity Over Complexity: The authors found that adding complex, randomly initialized meta-learning layers (like Relation or Induction Networks) often led to overfitting. Directly fine-tuning BERT while leveraging label semantics proved more effective and efficient.

Conclusion

LaSAML demonstrates that class labels are "free" features that should not be ignored. By treating labels as text rather than IDs, models can achieve substantial gains without complex architectural changes. However, the method's success depends on the semantic quality of class names. Future work may explore using full lexical definitions to provide even deeper guidance to meta-learners.

Summary updated

BibTeX
@inproceedings{luo2021don,
  author = {Qiaoyang Luo and Lingqiao Liu and Yuhao Lin and Wei Zhang},
  title = {{Don't Miss the Labels: Label-semantic Augmented Meta-Learner for Few-Shot Text Classification}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.245},
}
Figure