TL;DR
This paper benchmarks 27 computational tools for transferring cell type labels from single-cell RNA sequencing (scRNA-seq) to single-cell ATAC sequencing (scATAC-seq) data. They found that methods like Bridge and GLUE perform best when high-quality paired data is available, while bindSC and GLUE are top choices for unpaired data. The study also highlights how data characteristics and method design choices significantly impact performance.
What Problem Does This Paper Address?
Single-cell RNA sequencing (scRNA-seq) has been a workhorse for understanding cellular diversity. However, scATAC-seq, which measures chromatin accessibility, provides complementary information about gene regulation. Accurately annotating cell types in scATAC-seq data is crucial but challenging, especially when direct labels are scarce. While many tools exist for scRNA-seq annotation, fewer are specifically designed for scATAC-seq, and even fewer bridge the gap between scRNA-seq and scATAC-seq. This paper addresses the need for a comprehensive evaluation of existing tools to guide researchers in selecting the most effective methods for cross-omics label transfer.
Methods
The study evaluated 27 computational tools, categorizing them into those designed for direct label transfer and those for joint embedding of scRNA-seq and scATAC-seq data. For joint embedding methods, a kNN classifier was trained on the latent space derived from scRNA-seq data and then used to predict labels for scATAC-seq data. The researchers assessed performance using various accuracy metrics for both common cell types (present in both modalities) and ATAC-specific cell types (unique to scATAC-seq). Scalability was evaluated by measuring running time and peak memory usage across different data sizes. They also investigated the impact of data properties (imbalance, dissimilarity) and method properties (use of paired data, peak-level information, semi-supervised learning) on performance.
Datasets
The study utilized a substantial collection of 27 datasets, comprising both human and mouse tissues. These datasets were divided into three groups:
- Paired data: scRNA-seq and scATAC-seq measured from the same cells.
- Unpaired data (no corresponding paired data): scRNA-seq and scATAC-seq from the same tissue but not from the same cells, with no paired data available from that specific tissue.
- Unpaired data (with corresponding paired data): scRNA-seq and scATAC-seq from the same tissue, where paired data from that tissue was also available, allowing methods that utilize paired data as a "bridge" to be evaluated.
The authors also performed specific experiments by manually adjusting data imbalance (removing modality-specific cell types or balancing cell type compositions) to study its impact.
Key Insights and Ideas
- Paired data is a powerful bridge: When high-quality paired scRNA-seq and scATAC-seq data from the same tissue are available, methods like Bridge and GLUE (multi) significantly outperform others for label transfer.
- Peak-level information is key: Methods that leverage scATAC-seq peak-level data, rather than just gene activity scores, generally perform better, suggesting that raw accessibility information captures more nuanced biological signals.
- Data imbalance is a major hurdle: Datasets with a high proportion of modality-specific cell types or significant differences in cell type composition between scRNA-seq and scATAC-seq data lead to poor performance across most methods.
- Semi-supervised learning can be detrimental: Introducing semi-supervised training strategies, while intuitive, often negatively impacted performance for many methods, suggesting it's not a universally beneficial addition.
- Scalability varies greatly: Deep learning-based methods, particularly those using GPU and mini-batch training, tend to be more scalable in terms of time and memory, with methods like Bridge also showing good efficiency due to dictionary learning.
Key Results
- Top performers: For unpaired data, bindSC and GLUE achieved the highest overall prediction accuracy. When paired data was available, Bridge and GLUE (multi) were the best performers.
- ATAC-specific cells are challenging: Most methods struggled to accurately annotate ATAC-specific cell types, often misclassifying them into existing cell types. scGCN was noted for performing most similarly to a random classifier in this regard.
- Data binarization is generally unhelpful: Binarizing scATAC-seq data often decreased prediction accuracy, likely by removing important distributional differences between modalities.
- Scalability leaders: Methods like Bridge, GLUE, SCALEX, uniPort, and Cobolt demonstrated good scalability, with efficient running times and memory usage, especially for larger datasets. Conversely, methods involving matrix optimization or pairwise distance calculations (e.g., SCOT, Pamona, scMC) were less scalable.
Limitations and Future Work
The authors acknowledge that their study has limitations:
- Dataset diversity: While extensive, the collection of tissues and sequencing technologies was still limited.
- Ground truth bias: Cell type labels from original publications were used as ground truth, which may be subject to manual annotation biases.
For future work, they suggest developing methods that:
- Effectively handle data imbalance.
- Utilize peak-level ATAC data rather than solely gene activity.
- Properly incorporate high-quality paired data when available.
- Avoid over-normalization techniques like binarization.
- Use semi-supervised training cautiously.
- Incorporate prior biological knowledge, similar to GLUE's approach, to better inform cross-modal relationships.
Summary updated