Abstract
Predictive models trained on sequencing profiles often fail to achieve expected performance when externally validated on unseen profiles. While many factors such as batch effects, small data sets, and technical errors contribute to the gap between source and unseen data distributions, it is a challenging problem to generalize the predictive models across studies without any prior knowledge of the unseen data distribution. Here, this study proposes DeepBioGen, a sequencing profile augmentation procedure that characterizes visual patterns of sequencing profiles, generates realistic profiles based on a deep generative model capturing the patterns, and generalizes the subsequent classifiers. DeepBioGen outperforms other methods in terms of enhancing the generalizability of the prediction models on unseen data. The generalized classifiers surpass the state-of-the-art method, evaluated on RNA sequencing tumor expression profiles for anti-PD1 therapy response prediction and WGS human gut microbiome profiles for type 2 diabetes diagnosis.
Introduction
Biomedical research currently faces a significant "reproducibility crisis." Machine learning models trained to predict clinical outcomes, such as cancer drug responses, often suffer from a drastic performance drop when applied to independent patient cohorts. This phenomenon is primarily driven by batch effects—systematic variations arising from differences in laboratory protocols, equipment, and patient demographics. These variations create a "distribution shift" between the training (source) data and the real-world (unseen) data.
While domain adaptation techniques exist to mitigate these shifts, they typically require access to the target data distribution during training. In clinical settings, however, we often require domain generalization: the ability of a model to perform accurately on entirely novel data without prior exposure. DeepBioGen addresses this challenge by leveraging deep generative models to augment training data and improve model robustness.
Methods: The DeepBioGen Framework
DeepBioGen treats 1D sequencing data (e.g., gene expression or microbiome profiles) as visual patterns, enabling the use of powerful computer vision architectures.
Visual Pattern Formation
The first step involves grouping correlated features—such as genes within the same biological pathway—using k-means clustering. These features are then rearranged to create structured "visual" patterns. This transformation allows convolutional neural networks (CNNs) to exploit local dependencies that are often ignored by standard linear models.
Generative Augmentation
At the core of DeepBioGen is a Conditional Wasserstein GAN (cWGAN) with a gradient penalty to ensure training stability. The framework utilizes multiple generators initialized with different seeds to capture diverse "modes" of the biological distribution. Unlike traditional methods like SMOTE, which interpolate between existing data points, DeepBioGen’s GANs generate samples that expand the "frontier" of the source data. This boundary expansion effectively bridges the gap between the training distribution and the shifted distributions of unseen external studies.
Results
The efficacy of DeepBioGen was evaluated on two distinct biological challenges: predicting anti-PD1 immunotherapy response in melanoma (RNA-seq) and diagnosing Type 2 Diabetes (WGS gut microbiome).
Performance and Generalization
DeepBioGen-based classifiers (SVM and Neural Networks) consistently outperformed state-of-the-art predictors and other augmentation methods across both datasets. As shown in the performance curves, the AUROC for DeepBioGen (red line) remains superior and stable even as the augmentation rate increases up to 64x.
Clinical Impact
In the melanoma study, the DeepBioGen-SVM classifier demonstrated significant clinical utility. It identified a "Responder" group with a median survival time of 755 days, a substantial improvement over the 440 days predicted by the next best method, IMPRES. Furthermore, analysis of $\mathcal{H}$-divergence confirmed that DeepBioGen minimizes the distance between training and unseen test distributions more effectively than competing methods.
Discussion
DeepBioGen represents a significant step toward solving the domain generalization problem in genomics. By treating sequencing data as visual patterns and exploring the data "frontier" through GANs, it creates more robust predictive models.
However, limitations remain. The method's success depends on some level of feature or biological signal overlap between the source and unseen data. Additionally, the framework shows higher sensitivity with boundary-approximating models like SVMs and Neural Networks compared to ensemble methods like Random Forests. Future work will focus on automating hyper-parameter tuning and refining the visual pattern formation process to further enhance the power of computer vision architectures in genomic research.
Summary updated