Introduction
Missing data is a ubiquitous challenge in real-world machine learning, particularly within healthcare. Often, data is not missing at random but follows recurring patterns—for instance, due to varying clinical protocols across different hospitals. Standard approaches typically rely on imputation followed by regression or training independent models for each missingness pattern (Pattern Submodels, or PSM). However, imputation often fails when missingness is "informative" (Missing Not At Random), and PSM suffers from high variance when specific patterns have limited samples.
The SPSM Framework
The Sharing Pattern Submodel (SPSM) framework addresses this bias-variance tradeoff by allowing different missingness patterns to share information through a global "main model" while specializing only where necessary.
In SPSM, the coefficients for any given missingness pattern $m$ are decomposed into two components:
- Main Model (θ): A global set of weights shared across all patterns, acting as an "anchor" for general trends.
- Pattern Specialization (Δm): Local adjustments unique to a specific pattern.
The prediction is calculated as $f_m(x) = (\theta_{\neg m} + \Delta_{\neg m})^\top x_{\neg m}$. To ensure parsimony and prevent over-specialization, an $\ell_1$ penalty is applied to the specialization terms ($\Delta_m$). This forces the model to use non-zero specialization coefficients only when they significantly improve predictive performance.
Experimental Evaluation
The authors evaluated SPSM against several baselines, including XGBoost, Multi-layer Perceptrons (MLP), Ridge regression, and standard PSM. Testing was conducted across synthetic data (simulating informative missingness) and real-world datasets:
- ADNI: Neuroimaging and genetic data for Alzheimer’s prediction.
- SUPPORT: Survival scores for hospitalized patients.
- HOUSING: A non-healthcare dataset to test linear assumptions.
Key Results
A primary advantage of SPSM is its sample efficiency. By sharing information through the main model, SPSM stabilizes predictions for rare missingness patterns that lack sufficient data for independent modeling.
On the ADNI and SUPPORT datasets, SPSM matched or exceeded the performance of complex black-box models like MLPs and XGBoost. For example, in the ADNI regression task, SPSM achieved an $R^2$ of 0.66, equal to Ridge regression but with significantly higher interpretability. The sparse specialization terms allow researchers to identify exactly which features change in importance under specific missingness conditions.
Conclusion
SPSM bridges the gap between rigid global models and fragmented local ones. While currently limited to linear and logistic architectures, it provides a theoretically grounded and highly interpretable method for handling informative missingness. Future work may extend this regularized parameter-sharing philosophy to non-linear models.
Summary updated