DEFT-UCS: Data Efficient Fine-Tuning for Pre-Trained Language Models via Unsupervised Core-Set Selection for Text-Editing

Devleena Das, Vivek KhetanEMNLP 2024 · DOI

Abstract

Fine-tuning large-scale language models for specialized tasks like text editing typically requires vast amounts of annotated data. We present DEFT-UCS, a framework that utilizes Unsupervised Core-Set Selection (UCS) to reduce data requirements by up to 70% while maintaining or exceeding state-of-the-art performance. By leveraging unsupervised clustering and geometric sampling strategies, DEFT-UCS identifies high-utility "hard" samples that drive model generalization, offering a cost-effective path for domain-specific model adaptation.

Introduction

The prevailing paradigm in instruction-tuning Pre-trained Language Models (PLMs) emphasizes large-scale datasets. However, for nuanced tasks such as professional text editing—encompassing bias neutralization and coherence improvement—high-quality labels are expensive and scarce. Existing data pruning techniques often rely on supervised metrics, necessitating a pre-existing reference model. DEFT-UCS addresses this by demonstrating that the most informative training samples can be identified using only raw input text through unsupervised methods.

Methodology: Unsupervised Core-Set Selection

The core of our approach is the Unsupervised Core-Set Selection (UCS) algorithm. Unlike supervised methods, UCS operates on the latent representations of raw input sentences to identify a representative subset for fine-tuning.

Figure 1: The DEFT-UCS framework overview, illustrating the flow from raw data $D$ through the UCS algorithm to produce a core-set $D_c$ for fine-tuning the PLM.

The process involves four distinct stages:

  1. Embedding: Sentences are mapped to a vector space using Sentence-T5, which provides superior task separation compared to other embedding models.
  2. Clustering: K-Means clustering groups data into $K$ clusters, where $K$ corresponds to the known task intentions in the dataset.
  3. Geometric Sampling: Within each cluster, samples are categorized based on their cosine distance from the centroid. "Hard" samples (those furthest from the centroid) are prioritized as they represent diverse task boundaries and edge cases.
  4. Fine-Tuning: A Flan-T5 Large model is trained on a combination of a small stratified base set ($D_{base}$) and the selected core-set ($D_c$).

Experimental Setup

We evaluated DEFT-UCS using the CoEDIT dataset, which contains 82,000 samples. Performance was measured across eight benchmarks covering simplification (TurkCorpus, Asset), coherence/clarity (Iterator series), grammar (JFLEG), and neutralization (WNC).

Results and Discussion

DEFT-UCS demonstrates remarkable data efficiency. By training on only 32.5% of the original CoEDIT data, the model achieved comparable or superior SARI and ROUGE-L scores on 6 out of 8 evaluation datasets.

Figure 2: SARI scores across various benchmarks as a function of the percentage of the CoEDIT dataset used for training. DEFT-UCS (stars) consistently matches or outperforms the full-dataset baseline (diamonds) with significantly less data.

The Value of "Hard" Samples

A key insight from our study is that "hard" samples—those furthest from cluster centers—are significantly more valuable than "easy," prototypical samples when data is constrained. These outliers likely capture the complex linguistic boundaries necessary for robust generalization. Human evaluation confirmed this, with DEFT-UCS edits perceived as accurate 83.8% of the time, compared to 70.5% for the model trained on the full dataset.

Task-Specific Data Requirements

The utility of data pruning varies by task subjectivity. Objective tasks like simplification reached peak performance with just 12% of the data. In contrast, highly subjective tasks like neutralizing bias (WNC) required over 80% of the data to match the full-set baseline, suggesting that diversity in subjective tasks is harder to capture in small core-sets.

Conclusion

DEFT-UCS proves that unsupervised geometric clustering is a viable alternative to supervised data pruning. By reducing the reliance on massive labeled datasets, this framework enables more accessible fine-tuning for specialized NLP tasks. Future work will focus on automating hyperparameter selection for $K$ and sampling weights to further streamline the pipeline.

Summary updated

BibTeX
@inproceedings{das2024deft,
  author = {Devleena Das and Vivek Khetan},
  title = {{DEFT-UCS: Data Efficient Fine-Tuning for Pre-Trained Language Models via Unsupervised Core-Set Selection for Text-Editing}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.1132},
}
Figure