scCluBench: Comprehensive Benchmarking of Clustering Algorithms for Single-Cell RNA Sequencing

Ping Xu, Zaitian Wang, Zhirui Wang, Pengjiang Li, Jiajia Wang, Ran Zhang, Pengfei Wang, Yuanchun Zhou

Introduction

Single-cell RNA sequencing (scRNA-seq) has transformed our understanding of biological systems by enabling the profiling of gene expression at the level of individual cells. A fundamental step in scRNA-seq analysis is clustering, which groups cells into distinct types or states based on their transcriptomic profiles. However, the rapid proliferation of clustering algorithms—ranging from traditional statistical methods to advanced deep learning and graph-based models—has created a fragmented landscape. Researchers often lack standardized benchmarks to determine which tools are most effective for their specific biological contexts, particularly when dealing with the high sparsity and noise inherent in single-cell data.

The scCluBench Framework

To address these challenges, we introduce scCluBench, a standardized and comprehensive benchmarking framework designed to evaluate the performance of clustering algorithms across a wide range of scenarios. The framework is built on four pillars of standardization: curated benchmark resources, rigorous evaluation protocols, biological interpretation pipelines, and a unified modular workflow.

scCluBench Resources Overview

scCluBench evaluates 14 diverse models categorized into four groups:

  • Traditional: Louvain, Leiden, and SC3.
  • Deep Learning-based: scMAE, DEC, DESC, scDeepCluster, scNAME, scDCC, and scziDesk.
  • Graph-based: scCDCG, scGNN, and scDSC.
  • Foundation Models: scGPT and GeneFormer.

Dataset Diversity and Complexity

The benchmark utilizes 36 gold-standard scRNA-seq datasets covering 18 tissue types across human and mouse specimens. These datasets were selected to represent real-world complexity, including varying scales and high levels of missing data points.

Dataset Statistics and Tissue Distribution

Key characteristics of the benchmark datasets include:

  • Scale: Datasets range from small samples to "stress-test" sets with over 20,000 cells and 60,000 genes.
  • Sparsity: 34 of the 36 datasets exhibit sparsity rates exceeding 80%, reflecting the technical noise common in scRNA-seq.
  • Biological Breadth: Tissues include brain, lung, liver, pancreas, and more, ensuring the benchmark's relevance across diverse biological studies.

Key Results and Insights

Our comprehensive evaluation revealed several critical insights into the current state of scRNA-seq clustering:

  • Top Performers: The graph-based model scCDCG emerged as the overall top performer, demonstrating exceptional ability to maintain clear cluster boundaries and handle complex data structures. Among deep learning methods, scMAE (a masked autoencoder) proved to be the most robust, particularly in high-sparsity environments.
  • Foundation Model Performance: While biological foundation models like scGPT are powerful for general representation, they often underperform on task-specific clustering compared to specialized architectures. This suggests that "one-size-fits-all" models may still require task-specific optimization.
  • The GNN Trap: Many Graph Neural Network (GNN) models suffer from "embedding collapse," where different cell types become indistinguishable. scCDCG avoids this through a unique "cut-informed" mechanism.
  • Biological Prior Knowledge: Incorporating biological information, such as known marker genes, significantly enhances clustering accuracy. For instance, the DESC model saw a 28.1% improvement in accuracy when utilizing marker-overlap annotation compared to standard statistical mapping.

Conclusion

scCluBench provides a rigorous roadmap for the single-cell community, highlighting the strengths and weaknesses of current clustering technologies. While AI-driven approaches like scCDCG and scMAE are beginning to surpass traditional methods, challenges remain in scalability and stability. Future work will focus on expanding the framework to include multi-modal data and spatial transcriptomics, ensuring scCluBench remains at the forefront of single-cell analysis.

Summary updated

BibTeX
@inproceedings{xu2026scclubench,
  author = {Ping Xu and Zaitian Wang and Zhirui Wang and Pengjiang Li and Jiajia Wang and Ran Zhang and Pengfei Wang and Yuanchun Zhou},
  title = {{scCluBench: Comprehensive Benchmarking of Clustering Algorithms for Single-Cell RNA Sequencing}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i2.37110},
}
Figure