A penalized integrative deep neural network for variable selection among multiple omics datasets

Yang Li, Xiaonan Ren, Haochen Yu, Tao Sun, Shuangge MaQuant. Biol 2024 · DOI

TL;DR

This paper introduces a novel deep learning framework, PIN, for identifying important biological variables across multiple omics datasets. PIN addresses the challenge of integrating diverse datasets while accounting for both shared and unique biological signals, leading to more robust and stable variable selection compared to existing methods.

What Problem Does This Paper Address?

Analyzing omics data (like genomics, transcriptomics, etc.) is crucial for understanding complex biological processes and diseases. However, individual omics datasets are often small and high-dimensional, making it difficult to reliably identify the key biological variables (e.g., genes) that influence an outcome. While pooling datasets can increase sample size, it often overlooks the fact that the relationships between variables and outcomes can differ across studies or populations. Existing "deep feature selection" (DFS) methods struggle with small sample sizes, and traditional integrative analysis methods often assume linear relationships, which are too simplistic for complex biological data. This paper tackles the problem of performing robust variable selection across multiple, potentially heterogeneous, omics datasets by leveraging deep learning's ability to model complex, non-linear relationships and integrating information in a principled way.

Methods

The core contribution is a Penalized Integrative Deep Neural Network (PIN). PIN builds upon deep neural networks by incorporating a "variable-selection layer" that connects input variables to the first hidden layer. The weights in this connection layer are designed to represent variable importance.

Key methodological innovations include:

  • Integrative Variable Selection: Instead of analyzing each dataset independently or naively pooling them, PIN models each dataset with its own neural network and then applies penalties across the corresponding variable-selection weights from all networks. This allows for simultaneous variable selection across datasets.
  • Handling Homogeneity and Heterogeneity: The authors propose two specific PIN architectures:
  • HoPIN (Homogeneity Penalized Integrative Deep Neural Network): Designed for situations where important variables are largely consistent across datasets. It uses a group lasso penalty, meaning a variable is either important in all datasets or none.
  • HePIN (Heterogeneity Penalized Integrative Deep Neural Network): Designed for situations where important variables may overlap but also differ across datasets. It uses a sparse group lasso penalty, allowing a variable to be important in a subset of datasets.
  • Data-Driven Selection: A procedure is proposed to help researchers decide whether to use HoPIN or HePIN based on preliminary analysis of the data's characteristics.
  • Optimization: The method uses a smoothed approximation for non-differentiable regularization terms and employs gradient descent with momentum for optimization.

Datasets

The paper evaluates PIN on two types of real-world biological data:

  • Alzheimer's Disease (AD) Datasets: Three datasets from the Alzheimer's Disease Neuroimaging Initiative (ADNI) were used, comprising individuals categorized as cognitively normal (CN), mild cognitive impairment (MCI), and Alzheimer's disease (AD). The goal was to identify gene expression markers associated with cognitive decline, using the AD assessment scale 13 (ADAS13) as the outcome.
  • Ovarian Cancer Datasets: Four publicly available ovarian cancer datasets (TCGA, GSE26712, GSE32062, and GSE9891) were used to identify gene expression markers associated with patient survival.

The paper also describes the generation of simulated datasets for both homogeneity and heterogeneity scenarios, with varying numbers of variables (100 and 500) and different linear/nonlinear relationships.

Key Insights and Ideas

  • The "Curse of Dimensionality" in Omics is Exacerbated by Small Sample Sizes: Traditional methods struggle when the number of features (genes, etc.) far exceeds the number of samples, and this problem is amplified when trying to integrate multiple such datasets.
  • Heterogeneity is the Norm, Not the Exception: Biological signals can vary significantly across different studies, populations, or experimental conditions. A method that can gracefully handle these differences is crucial for accurate biological discovery.
  • Deep Learning for Non-linear Relationships: Omics data often exhibit complex, non-linear relationships that linear models cannot capture. PIN's deep learning architecture allows it to model these intricate patterns.
  • A Unified Framework for Integrative Variable Selection: PIN provides a principled way to perform variable selection that explicitly accounts for the structure of multiple datasets, offering a significant advantage over simply pooling data or analyzing datasets in isolation.

Key Results

  • Improved Variable Selection Stability: In both AD and ovarian cancer analyses, PIN (HoPIN for AD, HePIN for ovarian cancer) demonstrated higher "observed occurrence index" (OOI) values compared to traditional integrative analysis (Holnte/Helnte) and separate DFS (s-DFS) methods. This indicates PIN is more stable in identifying important variables across bootstrap replications.
  • Competitive Prediction Accuracy: PIN methods achieved comparable or better prediction accuracy (measured by Mean Squared Error - MSE) than existing methods in both real data analyses and simulations. For instance, in the AD analysis, HoPIN had an MSE of 0.026, similar to or better than others. In ovarian cancer, HePIN achieved an MSE of 0.030, matching s-DFS and outperforming Helnte.
  • Identification of Biologically Relevant Genes: The important genes identified by PIN in the AD and ovarian cancer studies were linked to relevant biological processes and diseases, suggesting the method's utility in biological discovery. For example, SLC45A1 (identified in AD) is linked to intellectual disorders, and PDHB (identified in ovarian cancer) is involved in malignant proliferation.
  • Simulation Performance: Simulation studies showed that PIN methods (HoPIN and HePIN) generally outperformed traditional methods (Holnte, Helnte, p-DFS, s-DFS) in terms of variable selection metrics (sensitivity, specificity, geometric mean, correct classification rate) across both linear and nonlinear scenarios, and for both homogeneity and heterogeneity situations.

Limitations and Future Work

  • Deciding Between HoPIN and HePIN: While a data-driven procedure is proposed, the authors acknowledge that deciding between homogeneity and heterogeneity situations can still be subjective. They suggest HePIN is a more cautious choice when the scenario is unclear, as it can also perform well in homogeneity settings.
  • Interpretability: While PIN identifies important variables, understanding the biological pathways through which these variables influence outcomes remains a challenge. Future work could involve integrating gene regulatory pathways into the PIN framework.
  • Handling Censored and Categorical Outcomes: The current PIN method is designed for continuous outcomes. The authors suggest extending it to handle censored data (as in survival analysis) and categorical outcomes, which are common in biological studies.

Summary updated

BibTeX
@article{li2024penalized,
  author = {Yang Li and Xiaonan Ren and Haochen Yu and Tao Sun and Shuangge Ma},
  title = {{A penalized integrative deep neural network for variable selection among multiple omics datasets}},
  journal = {Quant. Biol.},
  year = {2024},
  doi = {10.1002/qub2.51},
}
Figure