TL;DR
This paper introduces scE²TM, a novel single-cell embedded topic model that leverages external knowledge from foundation models and employs a new regularization technique to overcome "interpretation collapse." scE²TM achieves state-of-the-art performance in both cell clustering and biological interpretability, enabling more robust and meaningful insights from single-cell RNA sequencing data.
What Problem Does This Paper Address?
Single-cell RNA sequencing (scRNA-seq) has revolutionized our understanding of cellular heterogeneity, but analyzing this data computationally presents a challenge: balancing model performance with biological interpretability. Traditional methods often struggle to provide clear, distinct biological insights, a problem exacerbated by "interpretation collapse," where learned topics become redundant and fail to capture the full biological variation. The emergence of powerful single-cell foundation models offers an opportunity to integrate external biological knowledge, but effectively doing so within interpretable models remains an open question. This paper aims to address these limitations by developing a model that not only performs well in clustering but also provides highly interpretable and biologically meaningful representations of single-cell data.
Methods
scE²TM is built upon a Variational Autoencoder (VAE) framework and introduces two key innovations:
- Cross-View Encoder (CVE): This component integrates external knowledge from pre-trained single-cell foundation models (like scGPT) into the scE²TM embedding space. It uses a "cross-view mutual distillation" strategy to align transcriptomic data with external embeddings, encouraging consistent topic assignments between neighboring cells across different views. This allows scE²TM to leverage rich contextual biological information without requiring joint clustering or label transfer.
- Embedding Clustering Regularization (ECR): To combat interpretation collapse and enhance topic diversity, ECR treats topics as cluster centroids and genes as samples. It uses an optimal transport approach to link topic embeddings with gene embeddings, compelling each topic to represent a distinct gene cluster. This ensures topics are unique and span diverse semantic spaces.
The model also incorporates a sparse linear decoder for reconstruction and an overall loss function that combines VAE, CVE, and ECR losses.
Datasets
The study utilized a comprehensive set of 20 scRNA-seq datasets spanning human and mouse, covering various tissues and technologies, with sizes ranging from hundreds to over 100,000 cells.
Additionally, two specific datasets were used for topic-based perturbation experiments:
- A PBMC dataset from Kang et al. (GSE96583) for studying interferon (IFN) stimulation.
- A melanoma dataset from Tirosh et al. (GSE72056) for identifying malignant-specific topics.
The paper does not introduce new benchmarks but rather evaluates scE²TM against existing state-of-the-art methods on these established datasets.
Key Insights and Ideas
- External knowledge is crucial: Integrating knowledge from large foundation models (via CVE) significantly improves both clustering performance and interpretability, demonstrating the power of leveraging pre-existing biological information.
- Interpretation collapse is a real problem: Existing methods often sacrifice biological interpretability for clustering accuracy. scE²TM's ECR component directly addresses this by ensuring topic distinctness and diversity.
- Clustering performance doesn't guarantee interpretability: The study highlights a surprising lack of correlation between high clustering scores and good interpretability, underscoring the need for dedicated interpretability assessment metrics.
- Topics can model dynamic cellular processes: scE²TM can effectively simulate cell-state transitions (like IFN response or malignant transformation) by perturbing topic activations, offering a powerful tool for hypothesis testing.
- Melanoma topics have prognostic value: Identified malignant-specific topics in melanoma data show significant associations with patient survival, suggesting their clinical relevance.
Key Results
- Superior Clustering: scE²TM achieved state-of-the-art clustering performance across 20 datasets, outperforming seven leading methods with an average Adjusted Rand Index (ARI) gain of 11.3% and Normalized Mutual Information (NMI) gain of 8.7% over the best competitor.
- Enhanced Interpretability: scE²TM demonstrated substantial improvements in interpretability metrics, including an 8.6% gain in Interpretation Purity (IP), 66.5% in Topic Quality (TQ), 27.5% in GSEA quality, and 16.1% in ORA quality compared to the second-best method.
- Robustness: The model showed stable performance across various neighborhood sizes and regularization weights, indicating robustness.
- Biological Discovery: In a human pancreas dataset, scE²TM identified topics related to insulin biosynthesis and beta-cell function. In PBMCs, it accurately simulated IFN-stimulated states. In melanoma, it identified malignant-specific topics that correlated with patient survival.
- Computational Efficiency: scE²TM offers competitive computational efficiency, outperforming some network-based methods in terms of GPU memory usage.
Limitations and Future Work
- Tissue/Cell-Type Specific Pathways: The authors suggest collecting tissue- or cell-type-specific pathway datasets to enable more accurate interpretability evaluation, as current metrics are often evaluated against all known pathways.
- Hierarchical Topic Structures: Future work could explore developing hierarchical topic structures to capture topics at multiple granularities, allowing for more refined interpretations focused on individual cell features.
- Multimodal Data Integration: With the rise of multimodal single-cell assays, extending scE²TM and its interpretability metrics to multimodal data is a promising direction for more comprehensive biological signal capture.
Summary updated