Abstract
We consider the problem of variable selection in high-dimensional settings with missing observations among the covariates. To address this relatively understudied problem, we propose a new synergistic procedure—adaptive Bayesian SLOPE—which effectively combines the SLOPE method (sorted $l_1$ regularization) together with the Spike-and-Slab LASSO method. We position our approach within a Bayesian framework which allows for simultaneous variable selection and parameter estimation, despite the missing values. As with the Spike-and-Slab LASSO, the coefficients are regarded as arising from a hierarchical model consisting of two groups: (1) the spike for the inactive and (2) the slab for the active. However, instead of assigning independent spike priors for each covariate, here we deploy a joint "SLOPE" spike prior which takes into account the ordering of coefficient magnitudes in order to control for false discoveries. Through extensive simulations, we demonstrate satisfactory performance in terms of power, FDR and estimation bias under a wide range of scenarios. Finally, we analyze a real dataset consisting of patients from Paris hospitals who underwent a severe trauma, where we show excellent performance in predicting platelet levels. Our methodology has been implemented in C++ and wrapped into an R package ABSLOPE for public use.
Introduction
In modern data science, researchers frequently encounter high-dimensional problems where the number of potential predictors ($p$) significantly exceeds the number of observations ($n$). While standard regularization tools like the LASSO are effective for finding sparse solutions, they are often plagued by two specific issues: a high rate of "false discoveries" (selecting irrelevant variables) and the "over-shrinkage" of large, important signals, which leads to biased parameter estimates.
These challenges are exacerbated when the data is incomplete. Traditional approaches often rely on listwise deletion—discarding any observation with a missing value—which not only wastes valuable information but also introduces significant bias. While some methods exist for high-dimensional selection with missing data, few effectively handle the $p > n$ regime while simultaneously controlling for false discoveries and providing unbiased estimates. This paper introduces Adaptive Bayesian SLOPE (ABSLOPE), a robust framework designed to address these limitations.
The ABSLOPE Framework
ABSLOPE is a synergistic Bayesian procedure that integrates the strengths of two powerful methodologies: Sorted $l_1$ Regularization (SLOPE) and the Spike-and-Slab LASSO (SSL).
1. SLOPE and FDR Control
Unlike the standard LASSO, which applies a uniform penalty to all coefficients, SLOPE penalizes coefficients based on their rank. Larger coefficients are subjected to more stringent penalties, a mechanism that mimics the Benjamini-Hochberg procedure. This allows for rigorous control over the False Discovery Rate (FDR).
2. Spike-and-Slab LASSO for De-biasing
The SSL component utilizes a mixture of two distributions: a narrow "spike" prior for irrelevant noise and a wide "slab" prior for actual signals. This hierarchical structure allows important variables to "escape" the heavy shrinkage applied to the noise, resulting in more accurate, unbiased estimates for the active predictors.
3. The Joint "SLOPE" Spike Prior
The core innovation of ABSLOPE is the deployment of a joint "SLOPE" spike prior. Instead of treating each covariate independently, the model ties variables together through their magnitude rankings. This joint prior ensures that the model naturally controls the rate of false positives even when the dataset is incomplete.
Computational Implementation
To handle missing values and the complexity of the hierarchical prior, the authors utilize a Stochastic Approximation EM (SAEM) algorithm. This approach treats missing covariates and variable inclusion indicators as latent variables, iteratively imputing them while updating model parameters.
Recognizing the need for scalability, the researchers also developed SLOBE, a faster variant that replaces stochastic sampling with deterministic expectations. Implemented in C++ via the Rcpp package, SLOBE allows for the analysis of datasets with hundreds of variables in a matter of seconds, making high-dimensional Bayesian selection computationally feasible.
Experimental Evaluation
Synthetic Simulations
The method was tested across various scenarios involving different levels of sparsity, signal-to-noise ratios, and correlation structures. The simulations specifically addressed Missing Completely at Random (MCAR) and Missing at Random (MAR) mechanisms. Results showed that ABSLOPE consistently maintained the FDR at the target level (e.g., 0.1) and demonstrated higher statistical power and lower estimation bias compared to standard LASSO or SLOPE with mean imputation.
Real-World Application: TraumaBase
The researchers applied ABSLOPE to the TraumaBase dataset, which contains records of 6,384 severely traumatized patients from Paris hospitals. The objective was to predict platelet levels—a critical factor for blood clotting—using 15 pre-selected variables that exhibited up to 60% missingness.
ABSLOPE identified a sparse set of predictors (including Age, Shock Index, and Lactate) that align with clinical knowledge. Notably, the model achieved predictive accuracy comparable to a Random Forest while offering the high interpretability required for clinical decision-making.
Conclusion and Future Work
ABSLOPE provides a robust solution for variable selection and parameter estimation in high-dimensional settings with missing data. By integrating missing values into the probabilistic structure of the model, it treats incomplete data as a feature of the Bayesian hierarchy rather than a pre-processing nuisance.
While the current framework assumes a Missing at Random (MAR) mechanism and is optimized for continuous covariates, future work will focus on extending the model to handle Missing Not at Random (MNAR) data and mixed data types (categorical and continuous). The methodology is currently available for public use via the ABSLOPE R package.
Summary updated