Abstract
As databases grow larger, missing values become an inevitable challenge for predictive modeling. While traditional statistics focuses on inferential accuracy, modern machine learning prioritizes predictive performance. This paper presents a systematic benchmark of missing-value strategies using 520,000 CPU hours across massive health databases. We compare native support for missing values with various imputation techniques, finding that native support (Missing Incorporated in Attribute, or MIA) provides a fast, robust, and high-performing alternative to computationally expensive imputation methods.
The Challenge of Missing Data in Health ML
Missing data is a pervasive issue in healthcare, often resulting from varied clinical practices, equipment differences, or patient stability. In predictive modeling, the goal is to forecast outcomes like mortality or diagnosis from incomplete records. Historically, researchers have relied on "imputation"—filling in the blanks—but this can be slow and may inadvertently mask clinical signals. For instance, the absence of a test result can itself be an informative indicator of a patient's status (Missing Not At Random, or MNAR).
Methodology
The study evaluated 12 strategies using Gradient-Boosted Trees (specifically scikit-learn’s HistGradientBoosting) across four diverse health databases: Traumabase, UK Biobank, MIMIC-III, and NHIS. These datasets covered 13 different prediction tasks with sample sizes ranging from 2,500 to 100,000.
The strategies were categorized into:
- Single Imputation: Using simple statistics (Mean, Median) or conditional models (Iterative Imputer, KNN).
- Missingness Indicators ("Mask"): Adding binary columns to indicate which values were originally missing.
- Native Support (MIA): Modifying the tree-building algorithm to handle missing values directly by choosing the optimal split direction for them.
- Multiple Imputation via Bagging: Training ensembles on multiple imputed datasets to account for uncertainty.
Performance and Accuracy
A key finding was that adding a "mask" improved nearly every imputation method, confirming that health data is frequently MNAR. However, sophisticated imputation did not always translate to superior prediction.
As shown in the performance analysis, the best overall predictive scores were achieved by Iterative Imputation + Mask + Bagging. However, the MIA strategy (native support) performed remarkably well, consistently ranking near the top without the need for additional indicator columns or complex pre-processing.
Computational Efficiency
While bagging methods offered the highest accuracy, they came with a massive computational penalty. On large datasets ($n=100,000$), methods like KNN became intractable, while bagging methods were 100 to 200 times slower than their non-bagged counterparts.
MIA emerged as the most practical winner, finishing in hours where other methods took days or weeks. It naturally captures the information provided by a "mask" without doubling the feature space, making it the most efficient "bang for your buck" in large-scale health informatics.
Conclusion
For predictive modeling on large-scale health data, native support for missing values (MIA) in tree-based models is the gold standard. It offers a robust, fast, and high-performing solution that avoids the pitfalls and computational costs of complex imputation. When imputation is necessary, researchers should always include missingness indicators to capture the informative nature of missing data.
Summary updated