When evaluating machine learning models, the research community heavily relies on standard benchmarks with fixed train-test splits. A fraction of a percentage point improvement on a benchmark is often enough to claim a new state-of-the-art (SOTA) architecture. However, a growing body of empirical research reveals a fragile reality: these performance gains are frequently artifacts of the specific data partition rather than genuine algorithmic progress. When the data is reshuffled or split differently, leaderboards often collapse or completely invert. This phenomenon is known as data split sensitivity.
The Illusion of Architectural Progress
Why do fixed splits deceive us? A single, static partition freezes a specific distribution of data quirks. As Kyle Gorman et al. point out in We Need to Talk about Standard Splits, fixed test sets often contain idiosyncratic distributions of words or structures that differ from the training set. Models can inadvertently overfit to these specific quirks. When evaluating part-of-speech taggers across multiple random partitions, the authors found that system performance rankings became highly unstable compared to the traditional standard split.
This instability spans across diverse domains. In graph representation learning, Oleksandr Shchur et al. demonstrate in Pitfalls of Graph Neural Network Evaluation that model rankings are highly sensitive to the specific nodes chosen for training and validation. By simply altering the random seed of the split, the relative performance of different Graph Neural Networks (GNNs) fluctuates wildly, making it difficult to distinguish true architectural innovation from a lucky data partition. Similarly, Luciano Vinas et al. show in Simplifying GNN Performance with Low Rank Kernel Models that variations in split strategies—such as balanced versus sparse splits—can artificially inflate accuracy scores, masking the true effectiveness of simpler baseline models.
Leaderboard Reshuffling Across Domains
The sensitivity to how data is divided can completely reshuffle model leaderboards, rendering single-split comparisons meaningless. For instance, in recommender systems, Zaiqiao Meng et al. show in Exploring Data Splitting Strategies for the Evaluation of Recommendation Models that changing the data splitting method can completely reorder the performance leaderboard of models. A top-performing model under one split strategy may drop significantly under another.
In software engineering, Pengyu Nie explores this in Impact of Evaluation Methodologies on Code Summarization. The author demonstrates that a model appearing superior in a cross-project setting can be outperformed by a completely different architecture when evaluated using a time-segmented approach that mirrors real-world software evolution.
The variance introduced by data splits can even dwarf the variance caused by random weight initialization. In bio-signal modeling, Eloy Geenjaar et al. note in CiTrus: Squeezing Extra Performance out of Low-data Bio-signal Transfer Learning that the performance variance across different test splits frequently exceeds the variance across random initialization seeds. This means a "better" model might just be a model evaluated on a more favorable slice of data.
Quantifying the Volatility
How large are these fluctuations? In text simplification tasks, Laura Vásquez-Rodríguez et al. show in The Role of Text Simplification Operations in Evaluation that different random partitions can cause performance swings of up to 7 SARI points. When the stakes in modern machine learning research are fractions of a percent, a 7-point volatility means standard benchmarks are often measuring the luck of the split rather than algorithmic progress. A similar vulnerability is highlighted by John Walker Orr et al. in Event Detection with Neural Networks: A Rigorous Empirical Evaluation, where rankings established on a standard split failed to hold when evaluated on alternative random splits of the same dataset.
Mitigating Split Sensitivity
To build robust machine learning systems, researchers are developing evaluation frameworks that explicitly account for split variance rather than relying on point estimates from a single partition.
Massive Resampling
One straightforward solution is massive resampling. In A Framework for Effective Application of Machine Learning to Microbiome-Based Classification Problems, Begüm D. Topçuoğlu et al. argue that repeating data splits multiple times (e.g., 100 times) is essential for obtaining robust performance estimates and preventing false conclusions based on unrepresentative subsets.
Signal-to-Noise Stabilization
To formalize this mathematically, Yan Xue et al. introduce a "Mixture Estimator" based on 3×2 cross-validation in We Need to Talk About Reproducibility in NLP Model Comparison. This method stabilizes model comparisons by explicitly tying reproducibility to the Signal-to-Noise Ratio (SNR) of performance estimators across different data distributions.
Derandomized Inference
Furthermore, in fields like novelty detection where data splitting introduces "algorithmic randomness," Meshi Bashari et al. propose using conformal e-values in Derandomized novelty detection with FDR control via conformal e-values. This method allows researchers to aggregate evidence from multiple data splits to produce stable, reproducible results while maintaining strict statistical control over false discovery rates.