GESTALT: Fast, Generalizable Histopathology Image Synthesis from Gene Expression with Pretrained Pathology Foundation Models

Felix Ho, Alexandros Graikos, Dimitris SamarasarXiv 2026 · arXiv

TL;DR

GESTALT is a new framework that synthesizes high-resolution histopathology images from spatial gene expression data by predicting the latent embeddings of pathology foundation models. By reusing pretrained visual knowledge instead of learning to render from scratch, GESTALT achieves state-of-the-art image quality, trains in a fraction of the time, and generalizes across different gene panels and tissue types.

What Problem Does This Paper Address?

Spatial transcriptomics (ST) is a powerful technology that measures gene expression while preserving the spatial arrangement of cells in a tissue. While researchers often try to predict gene expression from images, the inverse—generating a tissue image from a transcriptomic state—is equally valuable. It allows scientists to visualize the physical, morphological consequences of molecular changes, such as how specific cancer mutations manifest as distinct histological patterns.

However, existing methods like GeneFlow are computationally expensive. They require training a generative model from scratch for every new dataset or gene panel, failing to leverage the vast amount of visual information already captured by modern pathology foundation models (FMs). Furthermore, these methods often struggle to generalize to unseen patients or higher image resolutions.

Methods

The authors introduce GESTALT (Gene Expression Synthesis of Tissue Appearance via Latent Transfer), which reformulates image synthesis as a two-stage process:

  1. Mapping Expression to Embeddings: Instead of predicting pixels, a lightweight gene-token transformer predicts the image embedding that a pathology foundation model (like UNI2-h) would assign to a tissue tile. It uses a k-nearest-neighbor (k-NN) graph attention mechanism to model how a cell's local microenvironment influences its appearance.
  2. Rendering via Diffusion: A pretrained diffusion decoder called PixCell then takes this predicted embedding and renders it into a realistic H&E (Hematoxylin and Eosin) stained tissue image. The authors use LoRA (Low-Rank Adaptation) to efficiently fine-tune the decoder for specific datasets without retraining the entire massive model.
Figure 1. GESTALT. Each cell is encoded along the gene axis and pooled into one token, cells are mixed only with their k=8 nearest neighbors, and a query readout produces the tile vector z. A de-standardization head maps z into a frozen FM’s embedding space, which PixCell renders. Only the purple components are trained; training regresses ˆe onto the FM’s embedding of the same tile (Eq. (6))

Datasets

The model was evaluated on several high-quality spatial transcriptomics datasets:

  • Xenium Melanoma (C1, C2, P1): Standard benchmarks for multi-cell and single-cell expression-to-image synthesis.
  • HEST-Xenium (Lung and Breast): Used to test more challenging generalization scenarios, including slide-disjoint evaluations and cross-study transfers.
  • NCBI/GEO cohorts: Used for testing zero-shot transfer across different institutions and tissue sources.

Key Insights and Ideas

  • Visual Knowledge Reuse: The core "aha moment" is that we don't need to teach a model what "tissue" looks like every time we change the gene panel. By targeting the embedding space of a foundation model, GESTALT reuses a pre-existing "visual vocabulary," making training much faster and more robust.
  • Decoupling Prediction from Rendering: By separating the molecular-to-morphological mapping from the actual image generation, the authors can identify exactly where errors occur—whether the model failed to understand the gene expression or the renderer failed to produce a clear image.
  • Native High Resolution: Unlike previous methods limited to 256x256 tiles, GESTALT's architecture naturally scales to 1024x1024 resolution by using a grid of queries that share spatial context across sub-region boundaries.
  • Controllable "What-If" Synthesis: Because GESTALT is paired with a statistical model of spatial expression, it can generate paired transcriptomic fields and images with explicit control over cell type, density, and composition.
GESTALT enables high-resolution 1024x1024 synthesis across different Xenium panels.

Key Results

  • Superior Quality: GESTALT outperformed the previous state-of-the-art (GeneFlow) on perceptual metrics, improving the Fréchet Inception Distance (FID) by 21–32 points across three Xenium panels.
  • Efficiency: Training GESTALT takes roughly 2–3 hours on a single GPU, compared to the ~12 hours reported for GeneFlow on more powerful hardware. It also uses significantly fewer trainable parameters.
  • Generalization: The model showed impressive zero-shot transfer capabilities, successfully generating images for gene panels it hadn't seen during training.
  • Morphological Control: Qualitative results demonstrate that the model correctly responds to changes in cell density and type. For example, increasing the fraction of melanoma cells in a synthetic field results in larger, paler nuclei and more eosinophilic surrounding tissue, matching real biological observations.
Synthetic images generated by GESTALT correctly reflect changes in cell-type proportions.

Limitations and Future Work

While GESTALT is a major step forward, the authors note that performance still degrades under "hard" distribution shifts, such as transferring between entirely different studies or tissue sources. Additionally, the synthetic expression fields generated by their statistical model, while useful, currently lack the full structural variability found in real tissue.

Future work may involve exploring more sophisticated spatial modeling and further improving the robustness of the molecular-to-latent mapping to handle even more diverse clinical datasets.

Summary updated

Abstract

Spatial transcriptomics measures gene expression while preserving where each cell sits in the tissue. The inverse, what a given transcriptomic state looks like, is less explored but equally useful: it lets researchers visualize the morphological consequence of a molecular state and interpret a spatial assay's molecular calls against real tissue structure. GeneFlow, the current state of the art on the Xenium gene expression-to-image benchmark, trains a model from scratch, incurring a high cost on every new target dataset it is applied to. We instead propose GESTALT, which reformulates gene expression-to-image synthesis as predicting a pathology foundation model's image embedding. A lightweight gene-token transformer predicts this embedding from multi-cell gene expression, and a frozen or LoRA-adapted diffusion decoder (PixCell) renders it as an image. Because this visual knowledge is reused rather than relearned, GESTALT trains in a fraction of the time. In both multi-cell and single-cell settings, our method outperforms the SoTA on perceptual metrics across all three Xenium datasets. Beyond improved image quality, GESTALT natively synthesizes at 1024 x 1024, and is evaluated in three generalization settings: zero-shot transfer across gene panels and datasets, held-out patients and disease states, and transfer across studies and tissue sources. Coupled with a statistical model of spatial expression, GESTALT also provides, to our knowledge, the first paired synthesis of cell-resolved transcriptomic fields and tissue images with explicit control over cell type, density, and composition. Code is available at https://github.com/trongnhandev1011/Gestalt.
BibTeX
@misc{ho2026gestalt,
  author = {Felix Ho and Alexandros Graikos and Dimitris Samaras},
  title = {{GESTALT: Fast, Generalizable Histopathology Image Synthesis from Gene Expression with Pretrained Pathology Foundation Models}},
  year = {2026},
  eprint = {2611.01141},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01141},
}
Figure