Abstract
Single-nucleus RNA sequencing (snRNA-seq) has significantly advanced our understanding of the disease etiology of neurodegenerative disorders. However, the low quality of specimens derived from postmortem brain tissues, combined with the high variability caused by disease heterogeneity, makes it challenging to integrate snRNA-seq data from multiple sources for precise analyses. To address these challenges, we present scMamba, a pre-trained model designed to improve the quality and utility of snRNA-seq analysis, with a particular focus on neurodegenerative diseases. Inspired by the recent Mamba model, scMamba introduces a novel architecture that incorporates a linear adapter layer, gene embeddings, and bidirectional Mamba blocks, enabling efficient processing of snRNA-seq data while preserving information from the raw input. Notably, scMamba learns generalizable features of cells and genes through pre-training on snRNA-seq data, without relying on dimension reduction or selection of highly variable genes. We demonstrate that scMamba outperforms benchmark methods in various downstream tasks, including cell type annotation, doublet detection, imputation, and the identification of differentially expressed genes.
Introduction
Analyzing single-nucleus RNA sequencing (snRNA-seq) data from postmortem brain tissue is notoriously difficult due to three primary hurdles. First, postmortem samples often suffer from degradation and high technical noise, resulting in low data quality. Second, the "dropout" effect—where limited mRNA in a single nucleus leads to many zero expression values—makes it hard to distinguish true biological silence from technical failure. Finally, the high dimensionality of genomic data (over 20,000 genes) poses a computational scalability problem for traditional Transformer models, which have quadratic complexity. This often forces researchers to rely on a small subset of "Highly Variable Genes" (HVGs), potentially discarding vital biological signals.
Methods
The authors introduce scMamba, a pre-trained model built on the Mamba architecture (Selective State Space Models). Unlike Transformers, Mamba scales linearly with sequence length, allowing scMamba to ingest the expression levels of all ~19,000 genes simultaneously.
Key architectural innovations include:
- Bidirectional Mamba Blocks: Since gene expression relationships are not inherently causal, the model processes data in both directions to capture complex gene-gene interactions.
- Linear Adapter Layer: Instead of "binning" expression values into discrete categories, scMamba uses a linear layer to preserve continuous expression data, preventing information loss.
- Gene Embeddings: The model utilizes unique gene embeddings rather than standard positional encodings, helping the network learn the biological identity of each input.
- Masked Expression Modeling (MEM): The model was pre-trained on 1.6 million cells by masking 15% of non-zero gene expressions and tasking the model with predicting the missing values.
Datasets and Evaluation
The study utilized a massive collection of human brain snRNA-seq data for pre-training, totaling approximately 1.6 million cells across 10 published datasets. For downstream evaluation, five diverse datasets (Lau, Leng, Smajic, Zhu, and a custom "Jung" dataset) focusing on Alzheimer’s and Parkinson’s diseases were employed. scMamba was benchmarked against industry standards such as Seurat, SciBet, scBERT, and scHyena.
Key Insights
scMamba's ability to process the full gene set allows it to capture subtle signals from lowly expressed genes that are typically discarded. During pre-training, the model's internal representations naturally clustered cells by type and subtype, even without explicit labels. Furthermore, the learned gene embeddings placed known marker genes for specific cell types close to each other in vector space. The authors also demonstrated that treating gene expression as a continuous value via a linear adapter significantly outperforms discretization methods. In imputation tasks, scMamba effectively "denoised" the data, aligning cells from different batches based on biological similarity rather than technical artifacts.
Results
scMamba consistently outperformed benchmark methods across various tasks:
- Superior Annotation: The model achieved the highest F1 scores across 8 major cell types, 72 subtypes, and 127 fine-grained subclusters, significantly outperforming Seurat and scBERT in identifying rare cell populations.
- Robust Doublet Detection: It outperformed specialized tools like DoubletFinder and Scrublet, achieving higher recall and F1 scores in both in vivo and simulated tests.
- Accurate Imputation: In "nonzero imputation" tests, scMamba showed much lower Mean Squared Error (MSE) and higher Pearson correlation (reaching ~0.80) compared to MAGIC and DCA.
- Enhanced DEG Analysis: The model improved the reproducibility of Differentially Expressed Gene (DEG) analysis, recovering ~50% of original DEGs even when only 20% of the data was available.
Conclusion and Future Work
While scMamba shows remarkable performance, distinguishing between "true biological zeros" and "technical dropouts" remains an inherent challenge. Currently optimized for human brain tissue, future work will involve expanding this foundation model to a broader range of tissues and species. Additionally, exploring the integration of multi-omics data, such as chromatin accessibility, within the Mamba framework is a promising next step for creating a universal single-cell foundation model.
Summary updated