The Quality Diversity Tradeoff in Data Selection

61 concepts51 papers1 proposal

When training modern machine learning models, the "data recipe"—the specific mix of examples used during training—is often more important than the model architecture itself. To optimize compute and accelerate training, a natural instinct is to filter out the noise and train only on the "best" data.

However, defining "best" purely by a single metric—such as high quality, high difficulty, or high similarity to a target domain—introduces a fundamental trap. The highest-scoring samples tend to look remarkably alike. By greedily selecting the top-rated data, we inadvertently collapse the variance of the dataset, leading to severe semantic redundancy. This tension is known as the quality-diversity tradeoff: a balancing act between maximizing the informational value of individual samples and preserving the broad coverage of the entire distribution.

The Redundancy Trap and Non-Monotonic Performance

Why does pure quality filtering fail? When we score a dataset using a quality classifier or a reward model, the highest-scoring examples typically cluster together in the semantic space.

As shown by Sang Michael Xie et al. in Data Selection for Language Models via Importance Resampling, greedy selection based on individual scores exhibits "mode-seeking" behavior. Instead of covering the full target distribution, the selector repeatedly pulls samples from a narrow, high-scoring mode.

This causes a phenomenon known as non-monotonic performance. Hongyi He et al. observe in Learning from the Best, Differently: A Diversity-Driven Rethinking on Data Selection that training on a smaller, strictly filtered subset of the highest-ranked data can actually yield worse downstream model outcomes than training on a larger, lower-quality sample. When a model encounters the same high-quality patterns repeatedly, it suffers from diminishing returns. Strict quality filtering inadvertently strips away the linguistic and domain-specific variety required for broad generalization.

Pure Quality Filtering (Mode-Seeking):
[ Cluster A (High Quality) ] ---> Selects everything here (Redundant)
[ Cluster B (Mid Quality)  ] ---> Discards completely (Lost Variety)
[ Cluster C (Low Quality)  ] ---> Discards completely

Balanced Selection (Space-Filling):
[ Cluster A ] ---> Selects top representatives
[ Cluster B ] ---> Selects top representatives
[ Cluster C ] ---> Discards truly noisy samples, keeps unique patterns

This failure mode is especially visible in instruction tuning. Alexander Bukharin et al. demonstrate in Data Diversity Matters for Robust Instruction Tuning that optimizing solely for individual response quality scores creates a highly redundant training set. While the model learns to answer standard prompts beautifully, its lack of exposure to diverse phrasing causes it to fail catastrophically on difficult, "worst-case" outlier prompts at inference time.

The Geometry of the Tradeoff

We can visualize this tradeoff geometrically by treating the dataset as a high-dimensional manifold where two competing forces dictate the optimal selection:

Data selection, therefore, cannot be treated as a simple thresholding problem. It must be framed as a joint optimization problem where diversity acts as a regularizer. As Chi Zhang et al. demonstrate in Harnessing Diversity for Important Data Selection in Pretraining Large Language Models, intentionally incorporating lower-scoring but diverse data clusters acts as a vital regularizer, preventing the model from overfitting to specific patterns and boosting performance across broad downstream tasks.

Algorithmic Frameworks for Balanced Selection

To mathematically balance these competing forces, researchers have developed several distinct algorithmic paradigms:

Submodular Optimization and Graphs

Submodular functions naturally model the concept of "diminishing returns," making them ideal for data selection. If a specific concept is already well-represented in our selected subset, the marginal utility of adding a similar sample—even a very high-quality one—is low.

Linjun Zhou et al. applied this to few-shot learning in Learning to Select Base Classes for Few-Shot Classification, using a similarity ratio to balance target relevance with feature diversity. Similarly, Yicheng Chen et al. introduced MIG in MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Space, which maps instructions into a semantic space graph and iteratively selects data points that maximize global information gain, ensuring broad concept coverage from a tiny fraction of the data.

Stratified Sampling and Clustering

Instead of filtering data globally, another approach divides the data space into distinct semantic neighborhoods first, then samples from each neighborhood.

The LASER framework, developed by Paramita Mirza et al. in Stratified Selective Sampling for Instruction Tuning with Dedicated Scoring Strategy, scores data for difficulty and quality within localized clusters. By performing stratified sampling across these clusters, it ensures that every semantic region contributes its highest-quality representatives without allowing any single dominant mode to crowd out the others.

Orthogonal Decomposition

To prevent the redundancy that naturally occurs among top-tier data, Hongyi He et al. developed ODiS in Learning from the Best, Differently: A Diversity-Driven Rethinking on Data Selection. This method uses Principal Component Analysis (PCA) to break down data quality into independent, orthogonal dimensions. This allows researchers to select the best data from multiple distinct categories simultaneously, preserving variety across domains and linguistic styles.

Manifestations Across Learning Paradigms

The quality-diversity tradeoff is a universal challenge across machine learning domains:

Ultimately, the shift toward data-centric AI has revealed that data curation is not a simple matter of filtering out "bad" data. As Tingyu Xia argues in Rethinking Data Selection at Scale: Random Selection is Almost All You Need, at scale, data diversity can be more important than individual sample quality. Successful data selection mechanisms must treat datasets as holistic distributions, ensuring that the pursuit of high-quality exemplars does not inadvertently blind the model to the rich variety of the real world.

Key Foundational Work

Learning to Select Base Classes for Few-Shot Classification (2020) — contributed 1 concept elements to this direction.
FMore: An Incentive Scheme of Multi-dimensional Auction for Federated Learning in MEC (2020) — contributed 1 concept elements to this direction.
Learning to select data for transfer learning with Bayesian Optimization (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_quality_diversity_tradeoff_66efa1e1,
  title = {quality-diversity tradeoff},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/quality-diversity-tradeoff/rdr_66efa1e1977d4806be317f2b8e8ba791},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_quality_diversity_tradeoff_66efa1e1_v1,
  title = {quality-diversity tradeoff},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/quality-diversity-tradeoff/rdr_66efa1e1977d4806be317f2b8e8ba791/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake