Abstract
Spatial transcriptomic (ST) imaging and sequencing data enable us to link tissue morphological features with thousands of previously unseen gene expression values, opening a new horizon for understanding tissue biology and achieving breakthroughs in digital pathology. Deep learning models are emerging to predict gene expression or classify cell types using images as the sole input. Such models hold significant potential for clinical applications, but require improvements in interpretability and robustness. We developed STimage as a comprehensive suite of models for both regression (predicting gene expression) and classification (mapping tissue regions and cell types) tasks. STimage is the first to thoroughly address robustness (uncertainty) and interpretability. For robustness, STimage predicts gene expression based on parameter distributions rather than fixed data points, allowing for generalisation at a population scale. STimage estimates uncertainty from the data (aleatoric) and from the model (epistemic) for each of thousands of imaging tiles. STimage achieves interpretability by analysing model attribution at a single-cell level, and in the context of histopathological annotation. Using diverse datasets from three cancers and one chronic disease, we assessed the model’s performance on in-distribution and out-of-distribution samples. STimage is robust to technical variations across platforms, data types, sample preservation methods, and disease types. Overall, STimage advances spatial transcriptomics by improving the prediction of gene expression from traditional histopathological images, making it more accessible for tissue biology research and digital pathology applications.
Introduction
Spatial Transcriptomics (ST) has revolutionized our ability to map gene expression within the structural context of tissue. However, the high cost and technical complexity of ST limit its routine clinical application. A major goal in digital pathology is to predict this molecular information directly from standard Hematoxylin and Eosin (H&E) stained images, which are inexpensive and ubiquitous. While previous deep learning models have attempted this, they often function as "black boxes," lacking interpretability and failing to generalize across different laboratory protocols or provide confidence measures for their predictions.
Methods
To address these challenges, we developed STimage, a comprehensive deep-learning framework that treats gene expression prediction as a robust regression and classification task. The core architecture utilizes a ResNet50 CNN backbone as a feature extractor.
Unlike traditional models that provide point estimates, STimage employs a Negative Binomial (NB) distribution layer to model gene expression counts. This approach accounts for the over-dispersion and sparsity inherent in transcriptomic data, allowing for the quantification of both aleatoric uncertainty (data noise) and epistemic uncertainty (model knowledge gaps). The pipeline also incorporates rigorous stain normalization and tissue masking to ensure robustness against technical artifacts.
For interpretability, STimage integrates Local Interpretable Model-agnostic Explanations (LIME) and Shapley Additive Explanations (SHAP), combined with nuclei segmentation. This enables researchers to identify specific morphological features, such as tumor cell nuclei, that drive the model's predictions.
Results
STimage was evaluated across diverse datasets, including 10x Genomics Visium (Fresh Frozen and FFPE), legacy ST technology, and clinical images from The Cancer Genome Atlas (TCGA).
Performance and Robustness
STimage consistently outperformed existing methods (e.g., STnet, Hist2ST) in Pearson correlation (Pcc) and spatial autocorrelation metrics. Notably, the model demonstrated superior performance on out-of-distribution (OOD) samples. For instance, it maintained a Pcc of 0.52 for the ATP1A1 gene when transitioning between different sample preservation methods, where other models often failed to generalize.
Biological Interpretability
The model's predictions were validated against expert pathological annotations. In TCGA datasets, predicted gene markers aligned closely with manual tissue regions identified by pathologists. Furthermore, single-cell attribution analysis confirmed that the model focuses on biologically relevant features, such as specific cell morphologies, rather than background noise.
Discussion
STimage represents a significant advancement in bridging the gap between histopathology and spatial transcriptomics. By providing uncertainty scores and interpretable visual maps, it offers a more reliable tool for clinical research. While current interpretability assessments are primarily qualitative, future work will focus on integrating automated cell segmentation with gold-standard cell type assessments. Additionally, optimizing the framework for ultra-high-resolution platforms like Xenium will be a priority to handle the computational demands of million-cell datasets.
Summary updated