60 Data Points are Sufficient to Fine-Tune LLMs for Question-Answering

Junjie Ye, Yuming Yang, Qi Zhang, Tao Gui, Xuanjing Huang, Peng Wang, Zhongchao Shi, Jiaqi Ji

Abstract

Large language models (LLMs) encode extensive world knowledge during pre-training, which can be activated for question-answering (QA) tasks through supervised fine-tuning (SFT). This study categorizes SFT data based on the extent of knowledge memorized by pretrained LLMs and conducts empirical analyses across four models from three model families. The results demonstrate that as few as 60 data points are sufficient to activate pre-trained knowledge for QA. Furthermore, the "memory level" of the SFT data significantly impacts performance, with training on well-remembered facts generally yielding the best results.

Introduction

While LLMs acquire staggering amounts of information during pre-training, they often require a "nudge" to output this knowledge in a structured QA format. The prevailing assumption has been that more SFT data is always better. This paper investigates three critical questions: the minimum data quantity required, the impact of data memory levels (how well the model already "knows" the training facts), and how these requirements vary across different model architectures like LLaMA and Qwen.

Methodology: Measuring Model Memory

To determine what a model already knows, the researchers introduced a Multi-Template Complementation Mechanism. Rather than testing a fact once, they designed 21 different mapping templates for every piece of knowledge (e.g., "{subject} is located in..." vs. "The country where {subject} is found is...").

The Multi-Template Complementation Mechanism uses various linguistic prompts to calculate a Memory Score based on response consistency.

By sampling the model 10 times for each template, they calculated a Memory Score ($R_k^M$) based on consistency across 210 trials. Data was then categorized into five levels, from $D_{train-0}$ (barely memorized) to $D_{train-4}$ (well-remembered).

Key Findings

The 60-Point Activation Switch

The most striking result is that SFT acts as an "activation switch" rather than a teaching process. Across LLaMA-2, LLaMA-3, and Qwen-2, peak or near-peak performance was achieved with only 60 training samples. This suggests the model isn't learning new facts during SFT, but rather learning the format of how to answer questions using its existing internal knowledge.

Test accuracy across different models and data sizes, showing that performance often plateaus or even declines after the initial 60 data points.

Quality Over Quantity

The study reveals that the "memory level" of training data is more important than the volume:

  • High-Memory Superiority: Training on facts the model already knows well ($D_{train-3}$ or $D_{train-4}$) consistently yielded the highest overall accuracy.
  • The Danger of Low-Memory Data: Training on facts the model has barely memorized ($D_{train-0}$) significantly impairs its ability to answer questions about things it does know well, likely confusing the model's internal retrieval system.
  • Diminishing Returns: Increasing SFT data from 60 to 960 or more often led to decreased performance, suggesting that excessive tuning can cause the model to overfit to specific samples and lose its broad generalization capabilities.

Conclusion

For practitioners, the implications are clear: massive SFT datasets for QA are often unnecessary and potentially counterproductive. Effective fine-tuning can be achieved with a very small, high-quality set of facts that the model already demonstrates a high degree of familiarity with. Future research will delve into the specific mechanical changes in transformer weights that allow such a small amount of data to activate latent parameters.

Summary updated

Abstract

Large language models (LLMs) encode extensive world knowledge through pre-training on massive datasets, which can then be fine-tuned for the question-answering (QA) task. However, effective strategies for fine-tuning LLMs for the QA task remain largely unexplored. To address this gap, we categorize supervised fine-tuning (SFT) data based on the extent of knowledge memorized by the pretrained LLMs and conduct a series of empirical analyses. Our experiments, involving four LLMs from three different model families, focus on three key factors: the amount of data required for SFT, the impact of different SFT datasets on model performance, and how data requirements vary across LLMs. The results show that as few as 60 data points during the SFT stage can activate the knowledge encoded during pre-training, enabling LLMs to perform the QA task. Additionally, SFT with data of varying memory levels has a significant impact on LLM performance, with the optimal dataset differing based on the specific model being fine-tuned. Future research will delve deeper into the mechanisms underlying these phenomena.
BibTeX
@misc{ye2024data,
  author = {Junjie Ye and Yuming Yang and Qi Zhang and Tao Gui and Xuanjing Huang and Peng Wang and Zhongchao Shi and Jiaqi Ji},
  title = {{60 Data Points are Sufficient to Fine-Tune LLMs for Question-Answering}},
  year = {2024},
  eprint = {2409.15825},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.15825},
}
Figure