Abstract
Multi-omics data, which integrate diverse high-dimensional molecular variables, are increasingly used to investigate disease outcomes. However, their utility for predicting survival time remains a subject of debate. This study presents a large-scale benchmark using 18 cancer datasets from The Cancer Genome Atlas (TCGA), comparing 11 machine learning and statistical methods—including boosting, penalized regression, and random forests. The study evaluates whether methods that account for the group structure of omics data outperform traditional clinical models. Results indicate that while structured methods slightly improve performance by protecting clinical variables from molecular noise, the overall utility of multi-omics data for survival prediction in these datasets remains limited. Only the block forest method marginally outperformed a standard Cox model on average.
Introduction
In the era of precision medicine, researchers have access to unprecedented volumes of biological data. For a single cancer patient, multi-omics profiles can include genomic mutations, gene expressions (RNA), copy number variations (CNV), and microRNA. The theoretical promise is that by integrating every molecular layer, we can predict patient survival with high precision.
However, this potential is often hindered by the "curse of dimensionality." In a typical TCGA dataset, the number of features ($p$) can exceed 100,000, while the number of observations ($n$) ranges from only 35 to 1,000. This $p \gg n$ scenario causes standard statistical models to fail. Furthermore, multi-omics data is inherently structured; features belong to specific biological layers. This study provides a rigorous benchmark to determine if complex models can truly outperform simple clinical variables like age and tumor stage.
Methodology
Data Sources and Processing
The benchmark utilized 18 different cancer types from The Cancer Genome Atlas (TCGA). For each cancer, five distinct data groups were merged:
- Clinical: Age, sex, and tumor stage (typically < 20 features).
- CNV: Copy number variations (~60,000 features).
- RNA: Gene expression (~20,000 features).
- Mutation: Somatic mutations (~15,000 features).
- miRNA: microRNA expression (~500–1,000 features).
Prediction Methods
Eleven methods were compared, categorized by their approach to the data's group structure:
- Lasso-based methods: Standard Lasso, IPF-Lasso (using different penalty factors per group), and Priority-Lasso (fitting groups successively based on priority).
- Statistical Boosting: Model-based boosting (mboost) and Likelihood-based boosting (CoxBoost).
- Random Forest: Standard implementations (
rfsrcandranger) and the structured Block Forest.
To handle the massive feature count, the study implemented "Two-Step" versions of IPF-Lasso and Priority-Lasso. These use a preliminary ridge regression to estimate group importance, making these methods computationally feasible for datasets with 100,000+ variables.
Evaluation Metrics
Performance was assessed using several repetitions of 5-fold cross-validation. The primary metrics were Uno’s C-index (measuring discriminative ability) and the integrated Brier score (measuring prediction accuracy).
Key Insights
The Clinical Anchor
Clinical variables are remarkably dense with information. A simple Cox model using only clinical data served as a high-performance baseline. The study found that "naive" machine learning methods—those that treat all 100,000+ features as a flat list—often performed worse than the clinical baseline. In these naive models, the predictive signal from clinical variables is frequently drowned out by molecular noise.
The Necessity of Structure
Methods that explicitly account for the multi-omics structure performed better. The most successful complex models were those that "favored" clinical data, including them without penalization and using molecular data only to explain residual variation. This approach ensures that known prognostic factors are not lost in the high-dimensional search space.
Performance Trends
- No Universal Winner: Across the 18 cancer types, no complex multi-omics method consistently and significantly outperformed the clinical-only Cox model.
- Block Forest and CoxBoost: Block forest was the only method to outperform the Cox model on average, though only slightly. CoxBoost (when favoring clinical variables) also proved highly competitive.
- The Sparsity Paradox: IPF-Lasso produced the most minimalist models by selecting the fewest features, while Priority-Lasso was more maximalist. Interestingly, the largest data group (CNV) was often the least useful for prediction.
- Cancer-Specific Success: While the average performance was flat, molecular data added significant value in specific cases, such as Low-Grade Glioma (LGG), where molecular markers are biologically known to be critical for prognosis.
Conclusion and Future Work
The study serves as a sobering reality check for multi-omics survival prediction. In many current TCGA studies, the added value of high-dimensional molecular data over simple clinical variables is marginal.
Future research should focus on:
- Computational Efficiency: Many promising methods, such as Sparse Group Lasso (SGL), remain computationally prohibitive for 100,000+ features.
- Structured Random Forests: Developing more sophisticated versions of Random Forest that can prioritize clinical data while capturing non-linear molecular effects.
- Sample Size: Increasing the number of observations to better match the dimensionality of the features.
Ultimately, for multi-omics to deliver on its promise, models must be designed to respect the hierarchy of biological information, treating clinical data as the foundation rather than just another set of variables.
Summary updated