On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation

Dan Iter, David GrangierarXiv 2021 · arXiv

Introduction

The standard "Pretrain then Fine-tune" pipeline has become the cornerstone of modern NLP. Models first learn general language features from massive, noisy datasets before specializing on smaller, high-quality "in-domain" datasets. However, generic pretraining data is often too broad. Data selection—filtering pretraining sets for domain-relevant examples—serves as a powerful intermediate phase to bridge this gap. This paper investigates the "complementarity" between data selection and final fine-tuning, revealing that aggressive selection can sometimes be counterproductive.

The Three-Phase Pipeline

The authors evaluate a training pipeline consisting of three distinct phases:

  1. Pretraining (PT): Training on a full out-of-domain (OOD) dataset.
  2. Selected Data Training: Using an intermediate classifier to rank OOD data and training further on the top-ranked subset.
  3. Fine-tuning (FT): Final training on the target in-domain dataset.

Two primary selection strategies are compared: Contrastive Data Selection (CDS), which uses likelihood differences between in-domain and OOD language models, and Discriminative Classifiers (DC), which range from Logistic Regression to fine-tuned BERT models trained to distinguish domain membership.

Loss curves comparing standard pretraining and fine-tuning against pipelines incorporating data selection (CDS and DC).

The Complementarity Trap

A key finding is the "complementarity trap": more aggressive data selection does not always yield better final performance. If a selection classifier overfits the small in-domain set, it selects data that is redundant with the fine-tuning set. This redundancy erodes the unique benefits of the final fine-tuning stage. Effective selection must find "near-domain" examples that provide novel information rather than just mimicking the fine-tuning data.

Experimental Results

The study covers Language Modeling (LM) on the Yelp corpus and Machine Translation (MT) for English-to-German (En-De) and English-to-Russian (En-Ru).

Machine Translation Performance

As shown in the results table, the BERT-based discriminative classifier consistently outperforms other methods. For En-De translation, the full PT + DC-BERT + FT pipeline achieved a BLEU score of 27.78, a significant improvement over the 26.89 BLEU achieved by standard fine-tuning alone. The BERT-based classifier outperformed the traditional CDS method with over 99% statistical significance across multiple metrics.

Machine translation performance (BLEU and logPPL) across English-German and English-Russian tasks.

Efficiency and Scaling

Data selection also offers a significant efficiency shortcut. Applying selection to an early pretraining checkpoint can reach target loss levels up to 3.5x faster than continuing with generic pretraining. Furthermore, there is a trade-off between selection size and speed: smaller sets (e.g., 50k sentences) offer rapid initial gains but lower performance ceilings compared to larger sets (e.g., 1 million sentences).

Conclusion

Discriminative classifiers, particularly those based on BERT, are superior to traditional contrastive methods for data selection. When properly balanced to avoid the complementarity trap, data selection and fine-tuning work together to significantly improve domain adaptation performance and training efficiency.

Summary updated

Abstract

Domain adaptation of neural networks commonly relies on three training phases: pretraining, selected data training and then fine tuning. Data selection improves target domain generalization by training further on pretraining data identified by relying on a small sample of target domain data. This work examines the benefit of data selection for language modeling and machine translation. Our experiments assess the complementarity of selection with fine tuning and result in practical recommendations: (i) selected data must be similar to the fine-tuning domain but not so much as to erode the complementary effect of fine-tuning; (ii) there is a trade-off between selecting little data for fast but limited progress or much data for slow but long lasting progress; (iii) data selection can be applied early during pretraining, with performance gains comparable to long pretraining session; (iv) data selection from domain classifiers is often more effective than the popular contrastive data selection method.
BibTeX
@misc{iter2021complementarity,
  author = {Dan Iter and David Grangier},
  title = {{On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation}},
  year = {2021},
  eprint = {2109.07591},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2109.07591},
}
Figure