Abstract
Missing values are unavoidable in many applications of machine learning and present challenges both during training and at test time. When variables are missing in recurring patterns, fitting separate pattern submodels have been proposed as a solution. However, fitting models independently does not make efficient use of all available data. Conversely, fitting a single shared model to the full data set relies on imputation which often leads to biased results when missingness depends on unobserved factors. We propose an alternative approach, called sharing pattern submodels, which i) makes predictions that are robust to missing values at test time, ii) maintains or improves the predictive power of pattern submodels, and iii) has a short description, enabling improved interpretability. Parameter sharing is enforced through sparsity-inducing regularization which we prove leads to consistent estimation. Finally, we give conditions for when a sharing model is optimal, even when both missingness and the target outcome depend on unobserved variables. Classification and regression experiments on synthetic and real-world data sets demonstrate that our models achieve a favorable tradeoff between pattern specialization and information sharing.
Introduction
Missing data is a perennial challenge in machine learning, particularly in domains like healthcare where clinical tests may be omitted based on a patient's perceived health status. Standard approaches typically involve either imputation—filling in missing values—or the use of pattern submodels (PSM), where a separate model is trained for every unique combination of missing variables.
However, both methods have significant drawbacks. Imputation often relies on the "Missing at Random" (MAR) assumption, which fails when missingness is informative (Missing Not At Random, or MNAR), leading to biased predictions. Conversely, independent pattern submodels are highly data-inefficient; in datasets with numerous missingness patterns, each submodel is trained on only a small slice of data, resulting in high variance and poor generalization. This paper introduces Sharing Pattern Submodels (SPSM), a novel approach that bridges the gap between a single "one-size-fits-all" model and independent submodels.
Methodology
The core innovation of SPSM is the assumption that while different missingness patterns may require specific adjustments, they share a common underlying "main" model.
For a given pattern $m$, the prediction is made using coefficients $(\theta + \Delta_m)$, where $\theta$ is the global shared parameter vector and $\Delta_m$ is a pattern-specific specialization. To enforce information sharing, the authors apply an $\ell_1$ penalty (Lasso-style regularization) to the specialization terms $\Delta_m$. This forces most adjustments to zero, effectively "sharing" the global parameters across patterns unless there is strong empirical evidence that a specific pattern requires a unique coefficient. The resulting optimization problem is convex and solvable using standard algorithms like L-BFGS-B for both regression and classification tasks.
Experimental Results
The authors evaluated SPSM against several baselines, including XGBoost, Multi-Layer Perceptrons (MLPs), and standard Ridge regression with imputation, across diverse datasets including ADNI (Alzheimer’s Disease Neuroimaging Initiative) and SUPPORT (hospitalized patient physiology).
Performance and Sample Efficiency
SPSM demonstrates superior sample efficiency compared to independent submodels. On the ADNI regression task, SPSM achieved an $R^2$ of 0.66, significantly outperforming the PSM baseline of 0.51. This highlights the power of parameter sharing when data per pattern is scarce.
Parsimony and Interpretability
A key advantage of SPSM is its interpretability. Because the specialization term $\Delta_m$ is sparse, the model provides a clear view of how missingness affects behavior. In the ADNI regression task, SPSM achieved top-tier performance using only 37 shared and 21 specialized coefficients, whereas independent models required 430 coefficients. This allows domain experts, such as clinicians, to identify exactly which variables change their influence when a specific test is missing, rather than interpreting a "black-box" imputation.
Conclusion
Sharing Pattern Submodels offer a robust, data-efficient, and interpretable framework for handling missing data. By leveraging sparsity-inducing regularization, SPSM captures the predictive signal inherent in missingness patterns while maintaining the generalization benefits of a shared global model. Across both healthcare and synthetic benchmarks, SPSM consistently matches or exceeds the performance of more complex non-linear models while providing the transparency of a linear system.
Summary updated