Abstract
Background: As databases grow larger, it becomes harder to fully control their collection, and they frequently come with missing values: incomplete observations. These large databases are well suited to train machine-learning models, for instance for forecasting or to extract biomarkers in biomedical settings. Such predictive approaches can use discriminative—rather than generative—modeling, and thus open the door to new missing-values strategies. Yet existing empirical evaluations of strategies to handle missing values have focused on inferential statistics.
Results: Here we conduct a systematic benchmark of missing-values strategies in predictive models with a focus on large health databases: four electronic health record datasets, a population brain imaging one, a health survey and two intensive care ones. Using gradient-boosted trees, we compare native support for missing values with simple and state-of-the-art imputation prior to learning. We investigate prediction accuracy and computational time. For prediction after imputation, we find that adding an indicator to express which values have been imputed is important, suggesting that the data are missing not at random. Elaborate missing values imputation can improve prediction compared to simple strategies but requires longer computational time on large data. Learning trees that model missing values—with missing incorporated attribute—leads to robust, fast, and well-performing predictive modeling.
Conclusions: Native support for missing values in supervised machine learning predicts better than state-of-the-art imputation with much less computational cost. When using imputation, it is important to add indicator columns expressing which values have been imputed.
Introduction
In the world of medical data, "missingness" is not merely a nuisance; it is often a feature. Whether it is a skipped lab test or an unrecorded vital sign, the absence of data often carries as much information as the data itself. Traditional statistical training focuses on inference—trying to understand the "true" underlying relationship between variables—where missing data is a hole that must be filled (imputed) to avoid bias. However, in modern machine learning, the goal is often prediction.
A new benchmarking study by Alexandre Perez-Lebel and colleagues challenges the traditional "impute-then-predict" pipeline, suggesting that for large-scale health databases, researchers should stop trying to fill in the blanks and instead allow models to handle the gaps natively.
Methodology
The researchers conducted a massive benchmark comparing 12 supervised predictive methods (with 36 total variations) across 13 different tasks. These tasks utilized four major health databases: Traumabase (TB), UK Biobank (UKBB), MIMIC-III, and the National Health Interview Survey (NHIS).
The study compared three primary strategies:
- Imputation-first approaches: Using Mean, Median, KNN, or Iterative Imputers to create a "complete" dataset before training.
- Native Support (Missing Incorporated Attribute - MIA): Using algorithms like Gradient-Boosted Trees (specifically
HistGradientBoostingfrom scikit-learn) that can natively decide how to branch a decision tree when a value is missing. - The "Mask" Technique: Adding a binary indicator column (a "mask") for every feature to explicitly tell the model which values were originally missing.
Results and Discussion
The benchmark results demonstrate a clear advantage for native handling of missing data. Gradient-boosted trees with native support consistently outperformed or matched state-of-the-art imputation techniques across various dataset sizes.
Informative Missingness
A key insight from the study is that missingness in health data is often informative, or "Missing Not At Random" (MNAR). For example, a clinician might not order a specific blood test because the patient appears healthy; the absence of that test is a signal in itself. The researchers found that adding a "mask" (indicator column) significantly improved the performance of simple imputation methods, as seen in the comparison between "Mean" and "Mean+mask" in the benchmark data.
Computational Efficiency
While elaborate imputation (like KNN or Multiple Imputation using Bagging) can improve results over simple mean imputation, it comes at a massive computational cost. As dataset sizes increased to 100,000 samples, some complex imputation methods became computationally "intractable." Native support, by contrast, is orders of magnitude faster. To put the scale of this study in perspective, the full benchmark required approximately 520,000 CPU hours.
Conclusion
Native support for missing values in supervised machine learning predicts better than state-of-the-art imputation with significantly less computational cost. Because the algorithm learns how to handle missingness during training, it captures the "signal" of the missing data without the noise of a guessed value. For practitioners who must use imputation (e.g., when using linear models), the authors conclude it is essential to add indicator columns (masks) to express which values were imputed.
Summary updated