Abstract
Preclinical perturbation screens, where the effects of genetic, chemical, or environmental perturbations are systematically tested on disease models, hold significant promise for machine learning-enhanced drug discovery due to their scale and causal nature. Predictive models can infer perturbation responses for previously untested disease models based on molecular profiles. These in silico labels can expand databases and guide experimental prioritization. However, modelling perturbation-specific effects and generating robust prediction performances across diverse biological contexts remain elusive. We introduce LEAP (Layered Ensemble of Autoencoders and Predictors), a novel ensemble framework to improve robustness and generalization. LEAP leverages multiple DAMAE (Data Augmented Masked Autoencoder) representations and LASSO regressors. By combining diverse gene expression representation models learned from different random initializations, LEAP consistently outperforms state-of-the-art approaches in predicting gene essentiality or drug responses in unseen cell lines, tissues and disease models. Notably, our results show that ensembling representation models, rather than prediction models alone, yields superior predictive performance. Beyond its performance gains, LEAP is computationally efficient, requires minimal hyperparameter tuning and can therefore be readily incorporated into drug discovery pipelines to prioritize promising targets and support biomarker-driven stratification.
Introduction
In precision oncology, the "holy grail" is predicting how a specific patient’s tumor will respond to a treatment before it is administered. To do this, scientists use high-throughput perturbation screens—massive experiments where thousands of drugs or genetic "knockouts" (using CRISPR) are tested on hundreds of cancer cell lines. However, translating this data into reliable predictive models is difficult due to challenges in generalizability across tissues, noise-induced reproducibility issues, and the "black box" nature of many high-performing architectures.
We present LEAP (Layered Ensemble of Autoencoders and Predictors), a framework that significantly improves the prediction of how cancer cells respond to drugs and gene knockouts. By ensembling diverse data representations and simple linear models, LEAP achieves state-of-the-art accuracy while being up to 150x faster than complex neural network alternatives.
Methods
The core innovation of LEAP is its two-layer ensembling strategy. Instead of just averaging different prediction models, LEAP averages different ways of looking at the data.
- Layer 1: Representation Ensembling. The framework uses Masked Autoencoders (MAE) to compress high-dimensional gene expression (RNAseq) data into lower-dimensional embeddings. Five different MAEs are trained using different random seeds; because deep learning is stochastic, each seed produces a slightly different "view" of the biological signal.
- Layer 2: Predictor Ensembling. For each of the five representations, five regressors are trained using 5-fold cross-validation, resulting in an ensemble of 25 models.
While several "heads" were tested, PS-LASSO (Perturbation-Specific LASSO) provided the best balance of performance and efficiency. To solve the interpretability problem, LEAP uses model distillation, training simpler, pathway-based models to identify which biological pathways (e.g., from the Reactome database) drive specific predictions.
Results
The study utilized massive datasets, including DepMap (CRISPR screens), PharmacoDB (drug response studies), and the PDX Encyclopedia (Patient-Derived Xenografts).
Key results include:
- Superior Accuracy: LEAP (PS-LASSO) outperformed the previous state-of-the-art (OmicsRPZ) by 16% in average Spearman correlation for gene dependency tasks.
- Massive Speedup: Compared to complex deep learning models (PP-tDNN) or gradient boosting (PP-LGBM), the PS-LASSO version of LEAP was 30x to 150x faster to train and run.
- Cross-Model Generalization: LEAP successfully predicted drug responses in PDX models, even though it was trained primarily on simplified cell lines, demonstrating its ability to bridge the gap to more "patient-like" models.
- Biological Grounding: The interpretability framework successfully identified known mechanisms, such as linking MAPK signaling pathways to the prediction of response for the MEK inhibitor refametinib.
Discussion and Future Work
LEAP demonstrates that stochasticity in neural networks can be a strength, capturing complementary signals from noisy RNAseq data. Ablation studies revealed that ensembling different data representations (the MAE outputs) contributed more to performance gains than ensembling different types of predictors.
While LEAP sets a new benchmark, limitations remain. Currently, the model only utilizes gene expression data; incorporating DNA mutations or proteomics could provide a more complete biological picture. Furthermore, while generalization to PDX models is improved, the lack of a tumor microenvironment in training data remains a hurdle for full clinical translation. Future work will explore more sophisticated "stacking" techniques to further boost ensemble performance.
Summary updated