Data-Efficient Finetuning Using Cross-Task Nearest Neighbors

Hamish Ivison, Noah A. Smith, Hannaneh Hajishirzi, Pradeep Dasigi

Abstract

Training large language models on every multitask dataset available is computationally expensive and can lead to negative interference, where irrelevant tasks degrade performance on a specific target. This paper introduces DEFT (Data-Efficient FineTuning), a method that leverages a small set of unlabeled target-task examples to retrieve only the most relevant labeled instances from a massive multitask pool. By training on approximately 2% of the available data, DEFT consistently outperforms models trained on the entire pool, demonstrating that targeted data selection is superior to the "kitchen sink" approach of instruction tuning.

Introduction

The paradigm of instruction tuning—training on a diverse collection of prompted tasks—has significantly improved the zero-shot generalization of large language models (LLMs). However, this approach faces two primary challenges: the immense computational cost of training on millions of examples and the risk of negative interference, where dissimilar tasks conflict during the learning process.

The authors of this paper propose a more surgical approach. Instead of training on the entire multitask pool, they ask whether a model can achieve better performance by focusing solely on data relevant to a specific target task. Crucially, they aim to identify this relevant subset without requiring expensive human-labeled examples for the new task.

Methods: The DEFT Framework

DEFT (Data-Efficient FineTuning) introduces an elegant workflow that avoids the need for target-task labels during the data selection phase. The process consists of four main stages:

  1. Indexing: A massive pool of prompted multitask data (such as P3 or SuperNaturalInstructions) is encoded into a vector space using a pretrained encoder (e.g., T5).
  2. Retrieval: Given a small set (32 to 1,000) of unlabeled examples from a target task, the system performs a $k$-nearest neighbor search to identify the most semantically similar labeled examples in the multitask pool.
  3. Targeted Finetuning: A pretrained model is finetuned exclusively on this retrieved subset.
  4. Optional Few-Shot Tuning: If a small number of labeled examples are available for the target task, a final round of parameter-efficient finetuning (using the IA3 method) is performed to create "DEFT-Few."
The DEFT workflow: from a pool of prompted tasks to a specialized model using unlabeled target instances.

A key innovation of DEFT is its instance-level selection. Unlike methods that select entire datasets, DEFT can pull specific, relevant examples from diverse sources, allowing it to "mine" a general pool for domain-specific knowledge even if the source datasets are not perfectly aligned with the target.

Experimental Setup

The researchers evaluated DEFT using two primary source pools:

  • P3 (Public Pool of Prompts): ~100 million examples used to train the T0 model.
  • SuperNaturalInstructions (SNI): 1,600+ tasks with expert-written instructions.

The target tasks included the 11 datasets used to evaluate T0 (e.g., RTE, ANLI, WinoGrande) and three specialized "diverse domain" datasets: CaseHold (legal), DROP (discrete reasoning), and Qasper (scientific NLP).

Results and Discussion

The experimental results demonstrate that "less is more" in the context of finetuning. DEFT-XL (3B parameters) outperformed the standard T0-3B model on 12 out of 14 datasets, achieving a median relative improvement of 13% while using only about 2% of the training data from the P3 pool.

Zero-shot performance of DEFT-XL compared to T0-3B and other baselines across various tasks.

Key Findings

  • Mitigating Negative Interference: The superiority of DEFT over full-pool training confirms that massive multitask pools contain significant "noise" for any specific task. Filtering for similarity effectively removes interfering data.
  • The Power of Unlabeled Data: Semantic similarity between inputs is a sufficient signal for data selection. This makes DEFT highly practical, as unlabeled data is often readily available.
  • Superior Initializations: DEFT models provide a better starting point for few-shot learning than general multitask models. When 20-70 labels were provided, DEFT-Few performed 7% better on average than T0-3B.
  • Domain Specialization: DEFT showed significant gains in specialized domains like legal (CaseHold) and scientific (Qasper) text, proving its ability to extract relevant knowledge from general-purpose pools.

Limitations and Future Work

While effective, DEFT has limitations. It requires at least a small set of unlabeled examples from the target task, which may not always be available. Furthermore, the current retrieval method focuses purely on similarity; incorporating diversity into the selection process could prevent the model from seeing redundant examples. Finally, the effectiveness of DEFT depends on the quality and coverage of the source pool; if the pool lacks relevant data, retrieval-based finetuning will be limited.

Future research could explore scaling DEFT to much larger models (70B+ parameters) and refining retrieval strategies to better balance similarity and diversity.

Summary updated

Abstract

Obtaining labeled data to train a model for a task of interest is often expensive. Prior work shows training models on multitask data augmented with task descriptions (prompts) effectively transfers knowledge to new tasks. Towards efficiently building task-specific models, we assume access to a small number (32-1000) of unlabeled target-task examples and use those to retrieve the most similar labeled examples from a large pool of multitask data augmented with prompts. Compared to the current practice of finetuning models on uniformly sampled prompted multitask data (e.g.: FLAN, T0), our approach of finetuning on cross-task nearest neighbors is significantly more data-efficient. Using only 2% of the data from the P3 pool without any labeled target-task data, our models outperform strong baselines trained on all available data by 3-30% on 12 out of 14 datasets representing held-out tasks including legal and scientific document QA. Similarly, models trained on cross-task nearest neighbors from SuperNaturalInstructions, representing about 5% of the pool, obtain comparable performance to state-of-the-art models on 12 held-out tasks from that pool. Moreover, the models produced by our approach also provide a better initialization than single multitask finetuned models for few-shot finetuning on target-task data, as shown by a 2-23% relative improvement over few-shot finetuned T0-3B models on 8 datasets.
BibTeX
@inproceedings{ivison2023data,
  author = {Hamish Ivison and Noah A. Smith and Hannaneh Hajishirzi and Pradeep Dasigi},
  title = {{Data-Efficient Finetuning Using Cross-Task Nearest Neighbors}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.576},
}
Figure