Spatial Transcriptomics Arena (STAr): an Integrated Platform for Spatial Transcriptomics Methodology Research

Xi Jiang, Danni Luo, Esteban Fernández, Jie Yang, Huimin Li, Kevin W. Jin, Yuanchun Zhan, Bo Yao, Suhana Bedi, Guanghua Xiao, Xiaowei Zhan, Qiwei Li, Yang XiebioRxiv 2023 · DOI

TL;DR

This paper introduces STAr, an integrated platform designed to address the challenges in analyzing spatially resolved transcriptomics (SRT) data. STAr provides a comprehensive resource of curated datasets, standardized analysis methods, and reproducible results, aiming to accelerate research and facilitate the unbiased evaluation of spatial gene detection techniques.

What Problem Does This Paper Address?

Spatially resolved transcriptomics (SRT) is a rapidly advancing field that allows researchers to study gene expression within its spatial context in tissues. A key goal in SRT analysis is identifying "spatially variable" (SV) genes, whose expression patterns vary across different locations in a tissue, offering insights into biological mechanisms and tissue heterogeneity. However, several hurdles hinder progress:

  • Data Fragmentation: Public SRT datasets are scattered across various websites and often preprocessed inconsistently, making it difficult for researchers to access and compare them.
  • Methodological Assessment: There's a lack of unbiased, comprehensive assessments of the numerous computational methods developed for SV gene detection, leaving researchers unsure which tools are best suited for their specific needs.
  • Reproducibility and Development: Implementing and comparing different existing methods for biological studies or developing new ones is cumbersome and time-consuming.

STAr aims to solve these problems by creating a centralized, user-friendly platform that consolidates SRT data, analysis tools, and results.

Methods

STAr is built around three core components:

  1. Curated Datasets: The platform houses a collection of 193 real SRT datasets from seven different technologies, along with simulated datasets generated under various spatial patterns and sparsity levels.
  2. Reproducible Statistical Analysis: STAr integrates seven established SV gene identification methods (BOOST-GP, BOOST-MI, BinSpect, SpaGCN, SPARK, SpatialDE, and Trendsceek) and two rank aggregation methods (GEO and MC2). These methods are applied to the curated datasets, and the analysis is made reproducible.
  3. Interactive Analysis Results: The platform offers a user-friendly interface to browse, compare, and visualize the analysis results, enabling users to explore spatial gene expression patterns and method performance.

The paper details the rigorous quality control applied to real datasets and the generative model used for creating simulated data, which mimics various spatial patterns and levels of zero counts (sparsity).

Datasets

STAr curates 193 real SRT datasets from 39 cohorts, spanning seven technologies:

  • Sequencing-based: ST, HDST, 10x Visium, STARmap, Slide-seqV2
  • Imaging-based: seqFISH/seqFISH+, MERFISH

These datasets cover 16 tissues from human and mouse, including brain, spinal cord, and ovaries, and are relevant to diseases like heart failure, melanoma, and breast cancer.

In addition to real data, STAr includes 600 simulated datasets. These were generated using five spatial patterns (spot, linear, MOB I, MOB II, BC) and four sparsity settings (0%, 10%, 30%, 50%), with 15 SV genes per dataset. This extensive simulation framework allows for a thorough benchmarking of SV gene detection methods.

Key Insights and Ideas

  • The Need for Standardization: The fragmentation and inconsistent preprocessing of SRT data are significant barriers to research. STAr's curated and standardized datasets are crucial for enabling fair comparisons.
  • Unbiased Method Evaluation is Critical: With a growing number of SV gene detection methods, a systematic and unbiased evaluation is essential to guide researchers toward the most appropriate tools for their specific applications.
  • Simulated Data is Key for Benchmarking: The paper highlights the importance of well-designed simulated datasets that capture realistic spatial patterns and sparsity to robustly assess method performance.
  • STAr as a Community Resource: The platform is designed to be extensible, encouraging contributions from the community to further expand its scope and impact.

Key Results

  • Comprehensive Dataset Collection: STAr successfully gathered and processed 193 real SRT datasets from diverse technologies and biological contexts.
  • Extensive Method Benchmarking: The platform provides a detailed comparison of seven SV gene detection methods across 20 combinatorial scenarios (five spatial patterns x four sparsity settings) using various performance metrics (AUC, F1 score, MCC).
  • Performance Differences Identified: The benchmarking revealed that methods like BOOST-GP, BOOST-MI, and BinSpect generally performed well, while others like SpatialDE and Trendsceek showed less satisfactory performance on simulated data. Specific methods were found to be better suited for certain spatial patterns (e.g., MOB I).
  • Computational Cost Analysis: The paper includes a comparison of the computational cost of different methods, noting that Bayesian methods (BOOST-GP, BOOST-MI) have a higher burden.
  • Interactive Visualization Tools: STAr offers powerful visualization capabilities, allowing users to explore gene expression patterns on tissue images and compare method results interactively.

Limitations and Future Work

  • Methodological Limitations: The paper acknowledges that some SV gene identification methods have limitations. For instance, kernel-based methods require pre-defined kernels, which might not capture all spatial patterns. Trendsceek was found to be computationally intensive with unsatisfactory performance.
  • Platform Evolution: STAr is designed as a dynamic resource. The authors welcome contributions of new datasets, methods, and analysis results from the community to continuously improve and expand the platform.
  • Future Research: The platform aims to accelerate future SRT methodological innovations by providing a robust and accessible resource for researchers. The ongoing curation of datasets and methods will facilitate a deeper understanding of biological processes and diseases.

Summary updated

Abstract

The emerging field of spatially resolved transcriptomics (SRT) has revolutionized biomedical research. SRT quantifies expression levels at different spatial locations, providing a new and powerful tool to interrogate novel biological insights. An essential question in the analysis of SRT data is to identify spatially variable (SV) genes; the expression levels of such genes have spatial variation across different tissues. SV genes usually play an important role in underlying biological mechanisms and tissue heterogeneity. Currently, several computational methods have been developed to detect such genes; however, there is a lack of unbiased assessment of these approaches to guide researchers in selecting the appropriate methods for their specific biomedical applications. In addition, it is difficult for researchers to implement different existing methods for either biological study or methodology development. Furthermore, currently available public SRT datasets are scattered across different websites and preprocessed in different ways, posing additional obstacles for quantitative researchers developing computational methods for SRT data analysis. To address these challenges, we designed Spatial Transcriptomics Arena (STAr), an open platform comprising 193 curated datasets from seven technologies, seven statistical methods, and analysis results. This resource allows users to retrieve high-quality datasets, apply or develop spatial gene detection methods, as well as browse and compare spatial gene analysis results. It also enables researchers to comprehensively evaluate SRT methodology research in both simulated and real datasets. Altogether, STAr is an integrated research resource intended to promote reproducible research and accelerate rigorous methodology development, which can eventually lead to an improved understanding of biological processes and diseases. STAr can be accessed at https://lce.biohpc.swmed.edu/star/ .
BibTeX
@misc{jiang2023spatial,
  author = {Xi Jiang and Danni Luo and Esteban Fernández and Jie Yang and Huimin Li and Kevin W. Jin and Yuanchun Zhan and Bo Yao and Suhana Bedi and Guanghua Xiao and Xiaowei Zhan and Qiwei Li and Yang Xie},
  title = {{Spatial Transcriptomics Arena (STAr): an Integrated Platform for Spatial Transcriptomics Methodology Research}},
  year = {2023},
  doi = {10.1101/2023.03.10.532127},
  howpublished = {bioRxiv},
}
Figure