The Spatial Accessibility Gap
Advanced spatial omics technologies, such as spatial transcriptomics (ST) and multiplexed immunofluorescence (mIF), have transformed our understanding of tissue architecture and the tumor immune microenvironment, as reviewed by Liu et al.. By mapping the precise locations of thousands of genes or dozens of proteins simultaneously, these tools reveal how cellular neighborhoods drive disease progression and treatment response.
However, these experimental methods face a severe accessibility gap. They are prohibitively expensive, suffer from low throughput, and require specialized hardware or complex antibody validation protocols that restrict them to well-resourced research core facilities, as noted by Migh.
Conversely, standard Hematoxylin and Eosin (H&E) histopathology slides are ubiquitous, rapid, and inexpensive to produce in routine clinical workflows. This stark asymmetry has inspired a powerful computational direction: virtual spatial omics. The goal is to train deep learning models to predict high-dimensional spatial molecular profiles directly from the visual features of standard, low-cost histology images, effectively using computational inference as a proxy for expensive physical sequencing.
Why Visual Features Encode Molecular States
At first glance, predicting the expression of thousands of individual genes from a simple brightfield image of stained tissue seems impossible. The reason this direction works relies on a fundamental biological principle: cellular morphology and tissue architecture are the physical manifestations of underlying gene expression programs.
A cell's shape, its nuclear-to-cytoplasmic ratio, its boundary interactions with neighbors, and the broader spatial arrangement of the extracellular matrix are tightly coupled to its functional state. For example, a tumor cell upregulating cell-cycle pathways will exhibit distinct nuclear atypia and crowding that a deep neural network can detect.
To successfully map these visual cues to molecular profiles, computational frameworks must capture features across multiple spatial scales simultaneously. Tissue biology is hierarchical; an individual cell's behavior is heavily influenced by its immediate neighborhood and the global organ architecture.
To exploit this hierarchy, Chung et al. developed TRIPLEX (Accurate Spatial Gene Expression Prediction by Integrating Multi-Resolution Features), a framework that explicitly integrates visual information across three distinct resolutions: individual cells, local neighborhoods, and global tissue architecture.
[H&E Slide] ───► Multi-Scale Feature Extraction
├── Cell Level (Nuclear morphology, edges)
├── Neighborhood Level (Cell-cell topology, hypergraphs)
└── Global Level (Tissue architecture)
│
▼
[Predicted Spatial Omics Map]
Furthermore, the boundaries and structural edges within a tissue contain dense regulatory signals. In BG-TRIPLEX (Boundary-Guided Learning for Gene Expression Prediction in Spatial Transcriptomics), Qu et al. demonstrated that guiding multi-branch attention mechanisms with explicit structural cues—such as cell edges and nuclear boundaries—significantly improves prediction accuracy.
By focusing on these morphological boundaries and modeling complex group relationships through hypergraphs and cross-modal contrastive learning, models like NH²2ST (Spatially Gene Expression Prediction using Dual-Scale Contrastive Learning) can faithfully translate local cellular topologies into accurate transcriptomic readouts.
From Regression to Generative Modeling
Early attempts at virtual spatial transcriptomics treated gene expression prediction as a standard pixel-to-value regression task. However, regression models suffer from a fundamental mathematical limitation in this domain: they optimize for the mean error, which forces the model to output "averaged" expression profiles. This smoothing effect obliterates the sharp biological heterogeneity and rare cell states that make spatial analysis valuable in the first place.
To preserve this vital biological variance, researchers have shifted toward generative modeling paradigms. Instead of asking "what is the exact average expression value for this pixel?", generative models learn the underlying probability distribution of the molecular data conditioned on the visual features.
For instance, Zhu et al. introduced Stem (Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images), which leverages conditional diffusion models paired with pathology foundation models. By framing the inference as a denoising process guided by H&E features, Stem captures complex biological heterogeneity without averaging it out.
Similarly, Huang et al. developed STFlow (Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching), utilizing whole-slide flow matching to model an entire tissue slide at once. This approach captures macro-level cell-cell interactions across gigapixel images that spot-by-spot regression methods miss entirely.
When scaled up to massive multi-organ datasets, foundation models like STPath (STPath: A Generative Foundation Model for Integrating Spatial Transcriptomics and Whole Slide Images) enable zero-shot prediction of tens of thousands of genes across diverse tissue types, demonstrating that the visual-to-molecular mapping generalizes far beyond a single organ.
Virtual Staining and Multiplex Generation
The same conceptual framework applies to spatial proteomics through a process known as virtual staining. Rather than physically treating a slide with successive rounds of expensive fluorescent antibodies, deep learning models can computationally synthesize high-dimensional multiplexed images directly from standard clinical stains.
This capability is realized at a whole-slide scale by frameworks like HistoPlexer, developed by Chen (Histopathology-based protein multiplex generation using deep learning), which transforms standard H&E images into high-dimensional 11-channel protein multiplex maps.
Similarly, Wu et al. developed ROSIE (ROSIE: AI generation of multiplex immunofluorescence staining from histopathology images), a generative framework that allows researchers to computationally "see" dozens of specific proteins and distinguish cell types—such as B cells and T cells—that are visually indistinguishable under standard brightfield illumination.
[Standard H&E Slide] ───► Generative Model (ROSIE / HistoPlexer) ───► [Virtual Multi-Channel Protein Map]
In scenarios where a baseline level of experimental data is available, these models can also act as super-resolution or upscaling tools. For example, the 7-UP framework developed by Wu et al. (7-UP: Generating in silico CODEX from a small set of immunofluorescence markers) takes a low-cost, standard 7-marker immunofluorescence panel and computationally expands it into a highly detailed 40-plex CODEX map by leveraging the underlying spatial and morphological context.
Grounding Predictions with Biological Priors
A major risk in virtual spatial omics is the potential for models to hallucinate plausible-looking but biologically impossible patterns, or to overfit to visual artifacts like slide preparation variations. Because deep neural networks are highly flexible function approximators, they can easily exploit spurious statistical correlations between pixel intensities and gene counts.
To ensure clinical utility, modern architectures constrain their predictions using prior biological knowledge. Cells do not express genes in isolation; genes operate within highly coordinated, evolutionarily conserved pathways and regulatory networks.
Hu et al. addressed this challenge in HistoPrism (HistoPrism: Unlocking Functional Pathway Analysis from Pan-Cancer Histology via Gene Expression Prediction) by introducing a "Gene Pathway Coherence" benchmark. This constraint ensures that the model's predicted gene expressions preserve known functional pathway relationships, preventing the model from chasing uncoordinated statistical noise.
Taking structural priors a step further, Shi et al. designed M2TGLGO (Multi-modal Topology-embedded Graph Learning for Spatially Resolved Genes Prediction from Pathology Images with Prior Gene Similarity Information). This framework embeds a pre-computed gene-to-gene similarity matrix directly into the topology of a graph neural network.
By forcing the network's internal representations to respect known biological relationships, the model is structurally discouraged from making unbiological predictions. These methods transform virtual spatial omics from a pure computer vision task into a biologically grounded paradigm capable of providing scalable, high-fidelity molecular insights from routine clinical slides.
Key Foundational Work
Single molecule chromogenic in situ hybridization assay for RNA visualization in fixed cells and tissues (2019) — contributed 2 concept elements to this direction.
Inference of the Xenopus tropicalis embryonic regulatory network and spatial gene expression patterns (2014) — contributed 1 concept elements to this direction.