2D, or not 2D? Investigating Vertical Signal Integrity of Tissue Slices

Sebastian Tiesmeyer, Niklas Müller‐Bötticher, Alexander Malt, Brian Long, Sergio Marco Salas, Paul Kießling, Paul Horn, Adrien Guillot, Louis Kümmerle, L.l. Ma, Frank Tacke, Fabian J. Theis, Christoph Kuppe, Mats Nilsson, Roland Eils, Naveed IshaquebioRxiv 2025 · DOI

TL;DR

This paper tackles a critical issue in spatial transcriptomics: the common practice of treating 3D tissue data as 2D, which can lead to inaccurate cell identification due to overlapping cells. They introduce ovrlpy, a novel computational tool that analyzes vertical signal integrity in 3D to identify and mitigate these "vertical doublets," improving the reliability of downstream analyses.

What Problem Does This Paper Address?

Spatial transcriptomics (SRT) technologies allow us to map gene expression within intact tissue sections, providing unprecedented insights into cellular organization. However, most current SRT studies, even those generating 3D data, often project this information onto a 2D plane for analysis. This simplification overlooks the vertical dimension of the tissue, leading to a significant problem: "vertical doublets." These occur when transcripts from cells stacked vertically in the tissue are conflated, causing a single segmented cell in a 2D view to contain genetic material from multiple distinct cells. This contamination can lead to misidentification of cell types and flawed biological interpretations. The paper aims to address this by developing a method to quantify and identify these vertical overlaps.

Methods

The core of the paper introduces ovrlpy, a computational tool designed to assess "Vertical Signal Integrity" (VSI). The strategy involves virtually splitting 3D tissue slices into "top" and "bottom" halves. This splitting isn't a simple global cut; instead, it uses a locally smoothed mean of transcript z-coordinates within spatial bins to create more comparable subslices.

ovrlpy then analyzes the signal consistency between these virtual subslices. It calculates local gene expression similarity, and regions with low VSI are flagged as potential overlaps. Crucially, ovrlpy is segmentation-free, meaning it doesn't rely on pre-existing cell segmentation masks, making it broadly applicable. It visualizes these low-VSI regions and can embed transcriptomic signals in both UMAP and RGB color spaces for interpretation.

Datasets

The study utilizes several publicly available datasets for evaluation:

  • Xenium mouse brain coronal section dataset: Used for demonstrating the ovrlpy workflow and identifying artifacts.
  • MERSCOPE mouse brain coronal section dataset: Used to validate the findings from the Xenium data and investigate technology-specific effects.
  • Publicly available mouse liver dataset (MERSCOPE platform): Used to assess the generalizability of ovrlpy across different tissue types.
  • 3D DAPI segmentation dataset of the mouse brain (Vizgen): Used to compare ovrlpy's findings with existing 3D nuclear segmentation models.

Key Insights and Ideas

  • The 2D projection is a significant source of error: The paper convincingly demonstrates that even with 3D data, projecting to 2D routinely contaminates signals, with a simplified model suggesting a low "signal-to-total" ratio for even ideal spherical cells.
  • Vertical doublets are common and cell-type dependent: The study shows that a substantial percentage of cells (e.g., ~22% in one mouse brain sample) are inconsistently assigned cell types between virtual top and bottom subslices, with glial and inhibitory neurons being particularly prone to forming vertical doublets.
  • ovrlpy offers a robust, segmentation-free solution: Unlike methods relying on cell segmentation, ovrlpy directly assesses signal integrity in 3D, making it more versatile and less susceptible to segmentation errors.
  • Tissue structure and preparation artifacts contribute to low VSI: Beyond cell stacking, tissue folds and inherent cellular morphologies (like in meninges or capillaries) also lead to low VSI, highlighting the multifaceted nature of signal contamination.
  • Existing 3D segmentation methods have limitations: The paper points out that even advanced 3D segmentation tools struggle to accurately identify vertically overlapping cells and tissue fold artifacts, underscoring the need for complementary quality control methods like ovrlpy.

Key Results

  • In a Xenium mouse brain dataset, approximately 22% of segmented cells showed inconsistent cell-type assignments between virtual top and bottom subslices.
  • In the hippocampal CA1 region of the same dataset, only ~20% of inhibitory/glial cells had consistent cell-type assignments between subslices, with the majority forming vertical doublets with neighboring neurons.
  • On a MERSCOPE mouse brain dataset, 83% overall consistency was observed, but this dropped to 51% for non-neuronal cells in the hippocampal CA1 region.
  • ovrlpy identified low-VSI regions that corresponded well with known tissue folds and areas with high cellular density or complex morphology.
  • Removing ovrlpy-identified vertical doublets led to improved signal-to-noise ratios and clearer cell-type separation in downstream analyses.

Limitations and Future Work

  • VSI threshold context-dependency: The paper acknowledges that the optimal VSI threshold for identifying doublets can vary depending on the tissue, cell type, and experimental conditions. Users need to perform context-specific evaluations.
  • Distinguishing sources of low VSI: While ovrlpy identifies low VSI, distinguishing between true vertical cell overlap, tissue processing artifacts (like folds), and other sources of signal heterogeneity (e.g., subcellular localization) requires careful examination of low-VSI regions.
  • Systematic evaluation against emerging 3D methods: The authors suggest that a systematic evaluation of ovrlpy's performance against newer 3D and multi-modal cell segmentation strategies is warranted.
  • Integration with existing tools: The paper highlights ovrlpy's compatibility with standard Python SRT infrastructure and its potential to complement existing 2D quality assurance tools. Future work could focus on deeper integration and broader adoption.

Summary updated

Abstract

Abstract Imaging-based spatially resolved transcriptomics can localize transcripts within tissue sections in 3D. Cell segmentation assigns transcripts to cells and precedes annotation of cell function. However, cell segmentation is usually performed in 2D, thus unable to deal with spatial doublets arising from overlapping cells, resulting in segmented cells containing transcripts originating from multiple cell types. Here we present a computational tool called ovrlpy that identifies overlapping cells, tissue folds, and inaccurate cell-segmentation by analyzing transcript localization in 3D.
BibTeX
@misc{tiesmeyer2025d,
  author = {Sebastian Tiesmeyer and Niklas Müller‐Bötticher and Alexander Malt and Brian Long and Sergio Marco Salas and Paul Kießling and Paul Horn and Adrien Guillot and Louis Kümmerle and L.L. Ma and Frank Tacke and Fabian J. Theis and Christoph Kuppe and Mats Nilsson and Roland Eils and Naveed Ishaque},
  title = {{2D, or not 2D? Investigating Vertical Signal Integrity of Tissue Slices}},
  year = {2025},
  doi = {10.1101/2025.01.13.632601},
  howpublished = {bioRxiv},
}

Loading related content...

Figure