Abstract
Training language models to learn from human instructions for zero-shot cross-task generalization has attracted much attention in NLP communities. Recently, instruction tuning (IT), which fine-tunes a pre-trained language model on a massive collection of tasks described via human-craft instructions, has been shown effective in instruction learning for unseen tasks. However, IT relies on a large amount of human-annotated samples, which restricts its generalization. Unlike labeled data, unlabeled data are often massive and cheap to obtain. In this work, we study how IT can be improved with unlabeled data. We first empirically explore the IT performance trends versus the number of labeled data, instructions, and training tasks. We find it critical to enlarge the number of training instructions, and the instructions can be underutilized due to the scarcity of labeled data. Then, we propose Unlabeled Data Augmented Instruction Tuning (UDIT) to take better advantage of the instructions during IT by constructing pseudo-labeled data from unlabeled plain texts. We conduct extensive experiments to show UDIT's effectiveness in various scenarios of tasks and datasets. We also comprehensively analyze the key factors of UDIT to investigate how to better improve IT with unlabeled data.
Introduction
Instruction Tuning (IT) has become a cornerstone of modern natural language processing, enabling models to generalize to unseen tasks by following natural language prompts. However, the effectiveness of IT is fundamentally limited by the availability of high-quality, human-annotated datasets. This "data bottleneck" makes it difficult to scale instruction following to niche domains or low-resource languages.
This research introduces UDIT (Unlabeled Data Augmented Instruction Tuning), a framework designed to bypass the need for expensive human labels. By leveraging the vast quantities of unlabeled text available online, UDIT allows models to learn the underlying structure of instructions through the creation of synthetic "pseudo-tasks."
Methods: The UDIT Framework
UDIT bridges the gap between raw text and structured instructions by using heuristic rules to transform plain text into pseudo-labeled training examples. This process allows the model to "practice" following instructions without requiring a human in the loop for every training point.
The framework focuses on eight common task clusters, including:
- Summarization: Treating document titles as summaries of the body text.
- Multiple-Choice QA: Generating distractors by replacing nouns in a sentence and asking the model to identify the original word.
- Topic Classification: Utilizing keywords found in source URLs (e.g.,
/sports/) to categorize the associated text. - Paraphrase Identification: Creating positive pairs via back-translation and negative pairs through word shuffling.
These pseudo-tasks are mixed with existing labeled data to fine-tune a 700M parameter T5 model. The primary objective is to help the model learn the semantic mapping between a natural language instruction and the expected output format.
Experimental Setup
The researchers utilized a 37.2GB multi-domain corpus including BookCorpus, CC-News, OpenWebText, and Wikipedia. Evaluation was conducted using the T0 benchmark, which includes 36 training datasets and 11 unseen evaluation datasets across categories such as Natural Language Inference (NLI) and Coreference Resolution.
Key Results and Insights
Scaling and Efficiency
A major finding of this study is that the diversity of instructions is often more critical than the volume of labeled samples. UDIT effectively increases this diversity by applying instructions to a wider variety of text domains. Notably, a 700M parameter model trained with UDIT can outperform a 3B parameter model trained using standard vanilla instruction tuning, demonstrating that data quality and diversity can effectively compensate for model scale.
Performance Gains
- Pure Zero-Shot: In settings with zero human-labeled training data, UDIT improved average accuracy from 43.0% to 49.6%.
- Few-Shot Scenarios: With only 100 labeled samples per task, UDIT boosted performance from 51.5% to 54.0%.
- Full Data Regularization: Even with access to all available human labels, UDIT provided a performance bump (56.9% to 57.8%), acting as a powerful regularizer that prevents overfitting to specific training domains.
Conclusion
UDIT demonstrates that unlabeled data is a viable and powerful resource for instruction tuning. By focusing on the relationship between instructions and task structures rather than just memorizing labels, UDIT achieves superior generalization with smaller models. Future work will explore automated instruction mining to reduce the reliance on manually designed heuristic rules and extend these methods to more complex reasoning tasks.
Summary updated