In traditional machine learning, a text classifier is a "black box" that learns to associate arbitrary numbers (labels) with patterns of words. To teach a model that "Label 1" means "Sports," you typically need to show it thousands of examples of sports articles. This creates a massive bottleneck: if you want to classify text into a new taxonomy, you must first embark on a months-long manual labeling campaign.
A growing body of research suggests a more intuitive path. Instead of treating labels as meaningless integers, we can treat them as semantic anchors. If a model already understands the meaning of the word "Sports," it should be able to find sports-related documents without seeing a single labeled example. This shift from supervised learning to "weakly supervised" or "dataless" classification leverages the latent knowledge already stored in pre-trained language models.
Labels as Semantic Anchors
The core intuition behind "dataless" classification is that both documents and category names can be mapped into the same high-dimensional semantic space. In On Dataless Hierarchical Text Classification, Yangqiu Song and Dan Roth demonstrated that by embedding labels and documents into a shared space—originally using Wikipedia concepts—a system can "understand" a label well enough to categorize text with zero training data.
This approach works because category names are not random; they are chosen specifically because they describe the essence of the documents they contain. Researchers like Yu Meng have extended this by using CatE (Discriminative Topic Mining via Category-Name Guided Text Embedding), which uses a few category names to guide the discovery of distinctive terms in a corpus. By modeling "word specificity," the system can distinguish between generic words and those that truly define a category.
From Seed Words to Classifiers
While a single label name is a powerful starting point, it is often too narrow to capture the full variety of a category. To bridge this gap, many researchers use a "bootstrapping" or "self-training" approach.
- Expansion: Start with a label name (e.g., "Politics") and find semantically similar words (e.g., "election," "senate," "policy"). Zied Haj-Yahia and colleagues showed that combining expert keywords with WordNet and word embeddings can create "category dictionaries" that act as a proxy for labeled data (Towards Unsupervised Text Classification Leveraging Experts and Word Embeddings).
- Pseudo-labeling: Use these keywords to find the "easiest" examples in an unlabeled dataset—documents that clearly belong to the category.
- Iterative Refinement: Train a model on these "pseudo-labeled" examples, then use that model to find more complex examples, gradually building a robust classifier.
A landmark example of this is LOTClass (Text Classification Using Label Names Only: A Language Model Self-Training Approach), where Yu Meng and Jiaxin Huang leveraged the masked language modeling (MLM) objective of BERT. By asking the model which words could replace a label in context, they automatically discovered relevant keywords and trained high-performing classifiers using only category names.
Leveraging the Power of Prompting
The rise of Large Language Models (LLMs) has simplified this process even further. Instead of complex bootstrapping, we can now use zero-shot prompting. In Large Language Models Are Zero-Shot Text Classifiers, Yiran Pang and Yanbin Lin found that models like GPT-4 can match or exceed the performance of specialized, fine-tuned models by simply being asked to classify a text.
However, LLMs can be "brittle" or biased. To solve this, Yunyi Zhang and Jiawei Han introduced PIEClass (PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training). PIEClass uses zero-shot prompting to generate initial labels but then applies a "noise-robust" ensemble training process. This allows the model to learn from the LLM's general knowledge while correcting for its specific mistakes using the patterns found in the actual unlabeled data.
Why It Works: The Structural Advantage
This research direction works because it stops treating text classification as a purely statistical task and starts treating it as a semantic matching task.
- Contextual Disambiguation: Methods like ClassKG (Weakly-supervised Text Classification Based on Keyword Graph) use Graph Neural Networks to understand that "windows" means software when it appears near "Microsoft," but a vehicle part when near "car."
- Hierarchical Awareness: In complex taxonomies, methods like HiLA (Label Augmentation for Zero-Shot Hierarchical Text Classification) use LLMs to automatically generate "child" labels, creating a denser semantic web that makes it easier to map documents to the correct level of a hierarchy.
By moving away from the "data-hungry" supervised paradigm, these methods allow us to deploy classifiers in niche domains, low-resource languages, and rapidly evolving fields where human experts are too expensive or too slow to keep up. The goal is no longer to label the data, but to describe the categories and let the model's pre-existing understanding of language do the rest.