Zero-shot learning (ZSL) has emerged as a powerful paradigm in Natural Language Processing (NLP), enabling models to categorize text into classes unseen during training. This capability hinges on the use of textual descriptions to define category semantics. However, recent research highlights a critical vulnerability: zero-shot models are highly sensitive to the specific phrasing of these descriptions. A minor lexical change—such as substituting "Company" for "Organization"—can significantly shift the model's decision boundaries, leading to inconsistent performance.
The Challenge: Description Sensitivity
The primary bottleneck in current ZSL workflows is the manual effort required to craft "optimal" descriptions. For non-experts, this often becomes a process of trial and error, as the internal logic of black-box models is difficult to predict.
As shown in Figure 1, the same mention can be classified differently based on the provided context and class definitions. This sensitivity gap suggests that zero-shot performance is as much a function of input semantics as it is of model architecture.
The UDEBO Framework
To address this, we introduce UDEBO (Unsupervised DEscription BOosting), an unsupervised framework designed to automate the generation and selection of diverse class descriptions. UDEBO operates in two distinct stages:
1. Automated Description Generation
UDEBO leverages four distinct strategies to create a diverse pool of descriptions, ensuring the model captures various semantic nuances:
- Pre-trained LMs: GPT-2 is utilized to expand brief descriptions into more comprehensive narratives.
- Fine-tuned LMs: A T5 model, fine-tuned on millions of Wikidata instances, generates structured and detailed definitions.
- Summarization: A BERT2BERT model distills noisy descriptions into concise, core definitions.
- Paraphrasing: A Pegasus model simplifies linguistic structures to improve generalizability.
2. Ranking and Ensembling
Once a set of candidate descriptions is generated, UDEBO employs an unsupervised ranking mechanism based on Information Entropy. Descriptions that result in lower entropy—indicating higher model confidence—are prioritized. Finally, the framework uses a majority voting ensemble across multiple classification pipelines to determine the final label, effectively triangulating the correct category through diverse semantic perspectives.
Key Results
UDEBO was evaluated across four benchmark datasets: OntoNotesZS and MedMentionsZS for entity classification, and FewRel and WikiZS for relation classification. The framework achieved new state-of-the-art (SOTA) results, demonstrating substantial improvements over existing baselines.
As detailed in Table 1, UDEBO significantly outperforms individual strategies and the standard ZS-BERT model across all metrics. Notable gains include a 7 percentage point improvement in Macro F1 on OntoNotesZS and a 6 percentage point boost on FewRel. While gains in specialized domains like MedMentionsZS were more modest (1.3 percentage points), the overall trend confirms that description ensembling is a robust strategy for zero-shot tasks.
Discussion and Conclusion
The success of UDEBO underscores the principle that diversity in input descriptions often provides more value than the precision of any single prompt. By using entropy as an unsupervised proxy for quality, UDEBO allows smaller, more efficient models to achieve performance levels previously reserved for massive, computationally expensive LLMs. Future work will focus on reducing the computational overhead of running multiple inference pipelines and exploring more sophisticated generation techniques to further refine the description boosting process.
Summary updated