In traditional Natural Language Processing (NLP), text classification is treated like a multiple-choice test where the options are arbitrary numbers or tokens. If you are classifying news articles, "Politics" might be mapped to Label 0 and "Sports" to Label 1. To a machine learning model, however, these indices carry no inherent meaning. The model is forced to learn the boundaries of "Label 0" entirely from scratch using explicit training examples.
As Chai et al. argue, this approach treats labels as semantically empty indices rather than informative instructions. A rapidly growing body of research flips this paradigm on its head: by replacing abstract identifiers with rich, natural language descriptions, we can transform classification from an arbitrary mapping problem into a semantic matching task.
The Problem with Semantically Empty Labels
When a model maps an input text to a discrete index or a one-hot vector, it discards a massive amount of prior knowledge. Pre-trained language models (PLMs) spend billions of tokens learning the intricate relationships between words. They already possess a deep understanding of what concepts like "politics," "symptoms," or "sports" mean.
When we force a model to map text to an abstract token like [CLS_1] or an integer like 0, we introduce a "data hunger" problem. The model cannot infer what a new index means without seeing numerous examples of it. This makes zero-shot generalization impossible and severely limits how well a model can adapt to new domains.
Furthermore, abstract labels fail to capture the relationships between categories. For example, a model treating "Cardiology" and "Neurology" as separate integer IDs cannot inherently recognize that both fall under the broader umbrella of medicine. As Dou et al. note, a large majority of domain-specific knowledge is language-inferable—the names and descriptions of entities provide immediate linguistic cues to their underlying semantic meaning.
The Mechanism: Grounding Labels in Language
To exploit this pre-trained semantic space, researchers reformulate classification tasks so that the model evaluates human-readable descriptions of the target categories. This grounding typically happens through three main architectural paradigms:
1. Text-to-Text Generation and Verbalization
Instead of using a discrete classification head, generative models are trained to output the actual words representing the category. Sainz et al. and Chen et al. demonstrate this via "label verbalization," where abstract relation tags are mapped to descriptive natural language templates.
For instance, instead of predicting a numeric score, Fernandez et al. map essay grades to descriptive terms like "poor" or "excellent," allowing the model to leverage its qualitative understanding of performance levels. Similarly, Sun shows that training generative models on fluent natural language strings rather than rigid, synthetic tags yields superior performance because it aligns perfectly with the model's internal linguistic fluency.
2. Reading Comprehension and Entailment
Classification can also be framed as a verification task. Chai et al. reformulate text classification as a machine reading comprehension (MRC) task where the model reads an input text and evaluates whether a specific category description applies to it.
In multi-label settings, the authors of Sren Kierkegaard at SemEval-2023 Task 4: Label-aware text classification using Natural Language Inference pair input premises with handcrafted textual descriptions of human values. This reframes a complex multi-label problem into a series of binary Natural Language Inference (NLI) choices: does the text entail the definition of the label?
Traditional Approach:
Input: "The patient presented with acute chest pain."
---> [Model] ---> Output: Class 4
Description-Based Approach:
Input: "The patient presented with acute chest pain."
Label Description: "A physical sign or symptom of a medical condition."
---> [Model evaluates compatibility] ---> Output: Entailment / Yes
3. Continuous Semantic Matching
Instead of relying purely on text generation, labels can be projected into a continuous vector space. Bapna et al. pioneered this by representing category labels as continuous vectors derived from their natural language descriptions.
Using a dual-encoder architecture, as seen in work by Chen, both the input text and the label descriptions are mapped into a shared semantic space. The model performs classification by computing the vector compatibility between the text representation and the description representation.
Why It Works: Intuition and Evidence
Shifting to natural language descriptions fundamentally changes how a model generalizes. It provides three distinct advantages:
- Zero-Shot and Few-Shot Generalization: Because the model understands the meaning of the description, it can identify entirely new categories at test time without any task-specific training data. Wang et al. demonstrate this in Named Entity Recognition (NER), where matching text spans against conceptual definitions allows models to recognize novel entity types simply by reading their descriptions.
- Cross-Domain Resilience: Traditional classification models break down during domain shifts because the vocabulary and label distributions change. However, Wang et al. and Cui et al. show that mapping tags to natural language descriptors makes models highly resilient to domain shifts, as the underlying semantic concepts remain stable across contexts (e.g., moving from news to medical text).
- Preventing Degenerate Outputs: In generative tasks, abstract tokens offer no structural guidance. Wang et al. show that incorporating structured semantic descriptions prevents models from defaulting to high-frequency generic phrases when encountering unfamiliar inputs, forcing the output to remain grounded in the target meaning.
Meaning vs. Structure
A natural skepticism arises: does this approach work because of the structural change in the task, or does the model truly utilize the linguistic meaning of the descriptions?
Li et al. designed an experiment to isolate this effect within an MRC framework for Semantic Role Labeling. They provided the model with explicit natural language definitions of task roles, and then ran a counterfactual test where those semantic descriptions were replaced with simple numeric identifiers.
The performance decreased significantly when the text descriptions were removed. This confirms that the model does not merely exploit the format; it actively processes and utilizes the semantic content of the definitions to guide its predictions. By treating labels as language, we allow models to do what they do best: read and understand.