Supervised Learning for Multi-Block Incomplete Data

Hadrien Lorenzo, Jérôme Saracco, Rodolphe ThiébautarXiv 2019 · DOI · arXiv

Abstract

In supervised high-dimensional settings characterized by a large number of variables ($p$) and a small number of individuals ($n$), selecting relevant variables while reducing dimensionality is a critical objective. However, the presence of missing data often compromises the validity of subspace selection. We propose Multi-block Data-Driven sparse Partial Least Squares (mdd-sPLS), a method designed to perform joint variable selection and subspace estimation while handling missing data through a novel imputation algorithm called Koh-Lanta. Benchmarked against existing methods such as mean imputation, NIPALS, softImpute, and imputeMFA, mdd-sPLS demonstrates the lowest prediction error for response variables. This approach is currently the only method that integrates supervised data imputation with response variable prediction. Its performance advantage increases with higher intra-block and inter-block correlations. Application to a real-world rVSV-ZEBOV Ebola vaccine trial dataset revealed biologically relevant predictors, demonstrating the method's efficacy in complex multi-omics contexts.

Introduction

Modern biological and medical research increasingly relies on "multi-omics" datasets, where multiple types of measurements (e.g., transcriptomics, proteomics, clinical markers) are collected from the same individuals. These datasets typically present two major challenges: high dimensionality ($n \ll p$) and missing data. Often, entire blocks of data—such as a specific assay at a certain time point—are missing for subsets of participants.

Traditional workflows address these issues in two disjoint steps: first, missing values are imputed using unsupervised methods; second, a predictive model is trained on the completed data. However, unsupervised imputation does not account for the relationship between covariates ($X$) and the response variable ($Y$), potentially introducing noise that degrades predictive accuracy. This paper introduces mdd-sPLS, a supervised framework that unifies imputation and prediction.

Methodology

1. The Foundation: ct-sPLS

The framework begins with Covariance-Thresholding sparse PLS (ct-sPLS). Unlike standard sparse PLS methods that rely on complex penalties (like LASSO), ct-sPLS utilizes a data-driven soft-thresholding operator. The regularization parameter, $\lambda$, is intuitively defined as the minimum correlation required between a covariate and the response variable for that covariate to be included in the model. This ensures that only variables with a direct, measurable relationship to the outcome are retained.

2. Multi-Block Extension (mdd-sPLS)

To handle multi-block data, mdd-sPLS calculates "super-weights" that aggregate information across different data sources. This allows the model to identify a shared subspace across blocks that best explains the variance in $Y$. Crucially, the method avoids "deflation"—a common PLS step that can become unstable with missing data—by constructing all axes simultaneously.

3. The Koh-Lanta Algorithm

The core of the missing data handling is the Koh-Lanta algorithm, a supervised imputation technique named after a survival reality show. It operates in two distinct stages:

  • The Tribe Stage: During training, an Expectation-Maximization (EM) based approach alternates between estimating the model parameters and using the current model to predict missing values in the training set.
  • The Reunification Stage: This stage handles missing values in the test set by applying the patterns and relationships learned during the Tribe Stage to ensure consistent predictions on new data.

Experimental Validation

Simulation Studies

The researchers conducted extensive simulations involving 10 blocks of covariates, varying the number of individuals, correlation strengths, and missing data percentages (ranging from 2% to 60%). The results showed that mdd-sPLS consistently achieved the lowest Root Mean Square Error of Prediction (RMSEP) compared to eight other methodologies. Notably, while the performance of competing methods deteriorated sharply as missingness approached 50%, mdd-sPLS remained stable, benefiting from the supervised nature of its imputation.

Ebola rVSV-ZEBOV Vaccine Trial

The method was applied to a challenging real-world dataset from an Ebola vaccine trial. The dataset included only 18 participants but featured over 18,000 gene expression variables and 129 cellular markers across 8 time-point blocks, with approximately 30% of the transcriptomic data missing.

mdd-sPLS successfully identified three specific genes—TIFA, SLC6A9, and FAM129B—as key predictors of the antibody response. These findings were validated against computationally intensive bootstrap studies, confirming that mdd-sPLS can efficiently identify biologically significant "needles in a haystack" even in extremely high-dimensional, sparse settings.

Conclusion

mdd-sPLS represents a significant advancement in supervised learning for incomplete multi-block data. By leveraging the response variable during the imputation process, it preserves the predictive signal that unsupervised methods often lose. The method is highly interpretable due to its correlation-based thresholding and is robust to high levels of missingness.

Availability: The method is implemented and available for the research community via the ddsPLS package on CRAN (for R users) and the py_ddspls package on PyPI (for Python users).

Summary updated

Abstract

In the supervised high dimensional settings with a large number of variables and a low number of individuals, one objective is to select the relevant variables and thus to reduce the dimension. That subspace selection is often managed with supervised tools. However, some data can be missing, compromising the validity of the sub-space selection. We propose a Partial Least Square (PLS) based method, called Multi-block Data-Driven sparse PLS mdd-sPLS, allowing jointly variable selection and subspace estimation while training and testing missing data imputation through a new algorithm called Koh-Lanta. This method was challenged through simulations against existing methods such as mean imputation, nipals, softImpute and imputeMFA. In the context of supervised analysis of high dimensional data, the proposed method shows the lowest prediction error of the response variables. So far this is the only method combining data imputation and response variable prediction. The superiority of the supervised multi-block mdd-sPLS method increases with the intra-block and inter-block correlations. The application to a real data-set from a rVSV-ZEBOV Ebola vaccine trial revealed interesting and biologically relevant results. The method is implemented in a R-package available on the CRAN and a Python-package available on pypi.
BibTeX
@misc{lorenzo2019supervised,
  author = {Hadrien Lorenzo and Jérôme Saracco and Rodolphe Thiébaut},
  title = {{Supervised Learning for Multi-Block Incomplete Data}},
  year = {2019},
  eprint = {1901.04380},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1901.04380},
}
Figure