Generation of super-resolution images from barcode-based spatial transcriptomics by deep image prior

Jeongbin Park, Seungho Cook, Dongjoo Lee, Jinyeong Choi, Seongjin Yoo, Sungwoo Bae, Hyung‐Jun Im, Daeseung Lee, Hongyoon ChoiCell Reports Methods 2024 · DOI

TL;DR

This paper introduces SuperST, a novel algorithm that leverages deep image priors to reconstruct high-resolution, dense gene expression maps from sparse barcode-based spatial transcriptomics data. SuperST significantly improves the resolution and accuracy of spatial gene expression visualization, offering a more comprehensive understanding of biological systems.

What Problem Does This Paper Address?

Spatial transcriptomics (ST) is a powerful technique for analyzing gene expression within its spatial context in tissues. However, current barcode-based ST methods, like 10x Visium, suffer from limitations: the data is sparse, meaning gene expression is only captured at specific "spots," leaving large areas of the tissue uncharacterized. This sparsity makes it challenging to reconstruct high-resolution images and interpret the underlying biology at a cellular level. Furthermore, the diffusion of RNA within tissues during sample preparation further limits the achievable resolution. Existing methods for improving ST resolution often rely on specific assumptions about gene expression distribution or struggle to accurately represent complex spatial patterns.

Methods

The core of SuperST is its reliance on a Deep Image Prior (DIP) approach. DIP is a deep learning technique that can reconstruct high-quality images from low-quality inputs (like noisy or blurred images) without requiring pre-labeled training data. SuperST utilizes an untrained neural network, specifically a U-Net, to constrain the image reconstruction process.

Here's a breakdown of the SuperST pipeline:

  1. Input: The process starts with a histology image of the tissue, which provides rich structural information. This image is resized to match the desired output resolution (e.g., 256x256).
  2. Deep Image Prior (DIP) with U-Net: The resized histology image is fed into a U-Net. The U-Net, guided by the DIP principle, learns to generate a high-resolution gene expression map.
  3. mRNA Diffusion Modeling: A convolution term using Gaussian smoothing is applied to account for the diffusion of mRNA in the tissue, ensuring that gene expression is realistically represented in the surrounding areas.
  4. Masking and Loss Function: The generated high-resolution image is then masked to retain only the pixels corresponding to the original ST spots. A loss function is calculated based on the difference between these masked pixel values and the actual gene expression values from the low-resolution ST data.
  5. Iterative Training: The U-Net is iteratively trained to minimize this loss function, effectively learning to predict the gene expression values for the dense matrix.

This approach differs from traditional methods by directly leveraging the structural information from histology images and employing a powerful generative model (DIP) that doesn't require extensive pre-training on specific gene expression datasets.

Datasets

The study utilized several datasets for evaluation:

  • Public Datasets:
    • 'Mouse Brain' (Rbfox3) from 10x Genomics.
  • 'Human Ovarian' (PTPRC) from 10x Genomics.
  • 'Human Ductal' (CD3G) from 10x Genomics.
  • A public breast cancer dataset (used for comparison with and without SuperST).
  • Newly Generated Dataset:
    • 'Mouse 4T1' (Pecam1) was generated by the authors from a 4T1-bearing mouse model. This dataset involved injecting tumor cells, collecting the tumor, and preparing tissue slices for ST library generation using the Visium protocol, along with H&E staining and immunofluorescence (IF) imaging.

Key Insights and Ideas

  • Histology as a Prior: The study highlights the significant value of using histology images as a "prior" for gene expression reconstruction. The structural context provided by histology helps guide the DIP model to generate more biologically plausible and spatially accurate gene expression maps.
  • DIP's Power for Sparse Data: Deep Image Prior is shown to be highly effective in reconstructing detailed images from sparse and low-resolution inputs, a common challenge in barcode-based ST.
  • Overcoming IF Limitations: While immunofluorescence (IF) images are a gold standard for visualizing protein expression, they can suffer from issues like autofluorescence and non-specific antibody binding. SuperST's outputs, by contrast, appear to be less affected by these artifacts, offering a cleaner representation of gene expression.
  • mRNA Diffusion is Key: Explicitly modeling mRNA diffusion provides a more realistic representation of gene expression patterns, moving beyond simple interpolation.

Key Results

  • Improved Correlation with IF: SuperST consistently achieved higher Pearson correlation coefficients between its generated gene expression maps and corresponding immunofluorescence (IF) images compared to conventional visualization methods and other super-resolution algorithms like XFuse and TESLA. For example, in the 'Mouse 4T1' dataset, SuperST achieved a correlation of 0.721 for Pecam1, significantly higher than conventional visualization (0.189).
  • Accurate Spatial Patterns: SuperST's output images more closely resembled IF images, accurately depicting spatial patterns and structures, such as the inner necrotic section and outer stromal compartment in the 'Mouse 4T1' sample.
  • Outperforms Existing Methods: Across multiple datasets and genes, SuperST demonstrated superior performance compared to XFuse and TESLA, which sometimes produced inappropriately contoured outputs or failed to generate results for certain genes.
  • Parameter Optimization: The study identified that the number of iterations ('num_iter') is a crucial parameter, with an optimal range balancing responsiveness to sparse data and avoiding overfitting.

Limitations and Future Work

  • Generalizability: The SuperST algorithm was evaluated on a limited number of samples, and further validation on a wider range of tissues is needed to confirm its generalizability.
  • Parameter Tuning: While 'num_iter' was explored, other parameters like image size, kernel size, and sigma value also require optimization for SuperST's performance.
  • Future Work: The authors suggest that SuperST has broad potential applications in various research fields that utilize ST. It could be integrated into other image processing workflows for ST data analysis and potentially combined with other spatial omics technologies (e.g., proteomics, metabolomics) for a more comprehensive understanding of biological systems.

Summary updated

Abstract

Spatially resolved transcriptomics (ST) has revolutionized the field of biology by providing a powerful tool for analyzing gene expression in situ. However, current ST methods, particularly barcode-based methods, have limitations in reconstructing high-resolution images from barcodes sparsely distributed in slides. Here, we present SuperST, an algorithm that enables the reconstruction of dense matrices (higher-resolution and non-zero-inflated matrices) from low-resolution ST libraries. SuperST is based on deep image prior, which reconstructs spatial gene expression patterns as image matrices. Compared with previous methods, SuperST generated output images that more closely resembled immunofluorescence images for given gene expression maps. Furthermore, we demonstrated how one can combine images created by SuperST with computer vision algorithms. In this context, we proposed a method for extracting features from the images, which can aid in spatial clustering of genes. By providing a dense matrix for each gene in situ, SuperST can successfully address the resolution and zero-inflation issue.
BibTeX
@inproceedings{park2024generation,
  author = {Jeongbin Park and Seungho Cook and Dongjoo Lee and Jinyeong Choi and Seongjin Yoo and Sungwoo Bae and Hyung‐Jun Im and Daeseung Lee and Hongyoon Choi},
  title = {{Generation of super-resolution images from barcode-based spatial transcriptomics by deep image prior}},
  booktitle = {Cell Reports Methods},
  year = {2024},
  doi = {10.1016/j.crmeth.2024.100937},
}

Loading related content...

Figure