TL;DR
This paper benchmarks 27 computational tools for transferring cell type labels from single-cell RNA sequencing (scRNA-seq) to single-cell ATAC sequencing (scATAC-seq) data. The study reveals that methods leveraging paired data and ATAC peak information perform best, while data imbalance and cross-omics dissimilarity significantly hinder accuracy. The findings offer crucial guidance for selecting and developing robust tools for cross-omics cell type annotation.
What Problem Does This Paper Address?
Accurately annotating cell types is fundamental to understanding the cellular composition of complex tissues. While scRNA-seq has been widely used for this purpose, scATAC-seq is increasingly employed to study regulatory genomic regions. However, annotating scATAC-seq data is challenging, with limited tools specifically designed for it. Many existing methods focus on label transfer within scRNA-seq or joint embedding of both modalities, which can then be used for label transfer. This paper addresses the need for a comprehensive evaluation of these tools to guide researchers in selecting the most effective methods for cross-omics label transfer from scRNA-seq to scATAC-seq.
Methods
The study evaluated 27 computational tools, categorizing them into those designed for direct label transfer and those for joint embedding. For joint embedding methods, a k-nearest neighbor (kNN) classifier was trained on the resulting latent space for label transfer. The authors also included kNN and random classifiers as baseline competitors. The performance was assessed using various accuracy metrics for common and ATAC-specific cell types, as well as scalability metrics like running time and peak memory usage.
Datasets
The study utilized a diverse collection of 27 datasets from both human and mouse tissues. These datasets were divided into three groups:
- Paired data: Where scRNA-seq and scATAC-seq were measured simultaneously from the same cell (9 datasets).
- Unpaired data with corresponding paired data: Where scRNA-seq and scATAC-seq were measured separately from the same tissue, but paired data from the same species and tissue were available (9 datasets).
- Unpaired data without corresponding paired data: Where scRNA-seq and scATAC-seq were measured separately from the same tissue, and no paired data was available (9 datasets).
The authors also conducted specific experiments to investigate the impact of data properties like imbalance and dissimilarity, and method properties like the use of paired data and semi-supervised training.
Key Insights and Ideas
- Paired data is king (when available): The availability of high-quality paired scRNA-seq and scATAC-seq data significantly boosts the performance of label transfer methods, with Bridge and GLUE (multi) emerging as top performers in such scenarios.
- ATAC peak information is crucial: Methods that utilize raw ATAC peak data, rather than solely relying on gene activity, generally achieve better cell type separation and accuracy.
- Data imbalance is a major hurdle: Datasets with a high proportion of modality-specific cell types or significant discrepancies in common cell type compositions lead to poor performance across most methods.
- Semi-supervised training isn't a universal win: Introducing semi-supervised learning can negatively impact performance for many methods, suggesting it's not a one-size-fits-all solution and its effectiveness is method-dependent.
- Deep learning excels at scalability: Deep learning-based methods generally demonstrate better scalability in terms of running time and memory usage, especially when leveraging GPU and minibatch training.
Key Results
- Best performers: When high-quality paired data was available, Bridge and GLUE (multi) were the top performers. In the absence of paired data, bindSC and GLUE achieved the highest overall prediction accuracy.
- ATAC-specific cell annotation remains challenging: No method performed exceptionally well on ATAC-specific cell types. scGCN showed performance closest to a random classifier, while scJoint tended to misclassify unobserved cell types with high confidence.
- Negative impacts: Data imbalance, cross-omics dissimilarity, data binarization, and semi-supervised training generally had negative impacts on model performance.
- Scalability: Bridge and deep-learning-based algorithms like GLUE were found to be the most time and memory efficient.
Limitations and Future Work
The authors acknowledge that their study has limitations, including the restricted variety of tissues and sequencing technologies covered. They also note that cell type labels derived from original publications might be subjective.
For future directions, they suggest that an ideal method should:
- Thoroughly address data imbalance.
- Incorporate ATAC peak-level information effectively.
- Properly leverage high-quality paired data when available.
- Avoid over-normalization techniques like binarization.
- Use semi-supervised training cautiously.
- Incorporate prior knowledge, such as regulatory interactions, to inform relationships between modalities.
Summary updated