Effective multi-modal clustering method via skip aggregation network for parallel scRNA-seq and scATAC-seq data

Dayu Hu, Ke Liang, Zhibin Dong, Jun Wang, Yawei Zhao, Kunlun HeBriefings in Bioinformatics 2024 · DOI

TL;DR

This paper introduces scEMC, a novel multi-modal clustering method for single-cell RNA sequencing (scRNA-seq) and single-cell Assay of Transposase Accessible Chromatin (scATAC-seq) data. It addresses the challenge of unbalanced information richness between these modalities by employing a skip aggregation network that prioritizes scRNA-seq data while effectively integrating scATAC-seq information, leading to improved clustering performance.

What Problem Does This Paper Address?

Single-cell sequencing technologies, like scRNA-seq and scATAC-seq, provide unprecedented insights into cellular heterogeneity. Analyzing these datasets together (multi-modal analysis) promises a more comprehensive understanding of cellular states. However, a significant challenge arises from the inherent differences in data quality and information content between these modalities. Specifically, scRNA-seq data typically contains richer information than scATAC-seq data, which can be sparse and noisy. Existing multi-modal clustering methods often treat these modalities equally, leading to the "garbage in, garbage out" problem where the less informative scATAC-seq data can degrade the overall clustering performance, sometimes even performing worse than using scRNA-seq data alone. This paper aims to bridge this gap by developing a method that effectively integrates these disparate modalities, leveraging the strengths of each while mitigating the weaknesses.

Methods

The core of scEMC lies in its Skip Aggregation Network (SAN) and a Zero-Inflated Negative Binomial (ZINB)-based denoising autoencoder.

  • ZINB-based Denoising Autoencoder: This module is designed to learn robust cell representations. It takes both scRNA-seq and scATAC-seq data as input, adds simulated Gaussian noise to each modality (to mimic real-world noise and improve robustness), and then uses an autoencoder to embed them into a lower-dimensional space. The ZINB distribution is employed to model the real distribution of single-cell data, accounting for over-dispersion and sparsity, which is particularly useful for scATAC-seq data. This module outputs three distributions (Dropout, Dispersion, and Mean) used for reconstruction loss.
  • Skip Aggregation Network (SAN): This novel component addresses the unbalanced information richness. It first concatenates the embeddings from both modalities. Then, inspired by transformer architectures, it maps these concatenated embeddings into multiple feature spaces. Crucially, it uses a skip connection to directly link the aggregated representation with the original scRNA-seq embedding. This mechanism ensures that the richer scRNA-seq information is preserved and guides the integration process, preventing the degradation of the overall representation due to sparse scATAC-seq data.

These two modules are jointly optimized using a combination of reconstruction loss (from the ZINB autoencoder) and a clustering loss (composed of KL divergence and deep k-means loss).

Datasets

The authors evaluated scEMC on five publicly available multi-modal single-cell datasets:

  • BMNC
  • PBMC
  • SLN111
  • SMAGE-10K
  • SMAGE-3K

These datasets were chosen because they contain both scRNA-seq and scATAC-seq data from the same batch of cells. For datasets that had undergone dimensionality reduction by their original authors, those processed forms were used. For others, feature counts were limited to 2000 to ensure consistency.

Key Insights and Ideas

  • Prioritizing Richer Modalities: The central insight is that not all data modalities are created equal. scEMC's skip connection mechanism explicitly acknowledges and leverages the superior information content of scRNA-seq data, preventing it from being diluted by sparser scATAC-seq data.
  • Transformer for Cross-Modal Structure: The use of a transformer-inspired architecture within the SAN module allows for learning global structural relationships across different feature spaces, enabling a more sophisticated fusion of information than simple concatenation.
  • ZINB for Realistic Data Modeling: The adoption of a ZINB distribution for the denoising autoencoder is a practical choice for modeling the often noisy and sparse nature of single-cell data, particularly scATAC-seq.
  • Ablation Studies Highlight Module Importance: The extensive ablation studies clearly demonstrate that both the skip connection and the aggregation mechanism are critical for scEMC's superior performance, underscoring the effectiveness of the proposed architecture.

Key Results

  • Superior Clustering Performance: scEMC consistently outperformed eight other baseline methods across five benchmark datasets, achieving the highest ARI (Adjusted Rand Index) and NMI (Normalized Mutual Information) scores in most cases. For instance, on the BMNC dataset, scEMC achieved an ARI of 0.6480 and NMI of 0.7603, significantly outperforming other methods.
  • Benefit of scATAC-seq Data: Figures 2 and 3 show that including scATAC-seq data (with scEMC) generally improves clustering performance compared to using scRNA-seq data alone, demonstrating that scEMC effectively integrates this modality.
  • Robustness to Hyperparameters: The study found that the NMI performance was relatively insensitive to the hyperparameters $\lambda_1$ and $\lambda_2$ (balancing reconstruction and clustering loss), while ARI showed some sensitivity, with optimal performance achieved at $\lambda_1=1$ and $\lambda_2=10$.

Limitations and Future Work

  • Limited Modality Scope: Currently, scEMC is designed for scRNA-seq and scATAC-seq data. The authors suggest integrating more sequencing modalities into the framework in the future.
  • Fusion Strategies: They propose exploring additional fusion strategies beyond concatenation and ensemble learning to further enhance aggregation capabilities.
  • Network Structure: The authors suggest that designing a more discriminative network structure could potentially improve the model's performance.
  • Computational Efficiency: While not explicitly stated as a limitation, the convergence analysis shows that the training process is terminated early once a threshold is met, implying a trade-off between full convergence and computational efficiency.

The code and datasets are publicly available at https://github.com/DayuHuu/scEMC.

Summary updated

BibTeX
@article{hu2024effective,
  author = {Dayu Hu and Ke Liang and Zhibin Dong and Jun Wang and Yawei Zhao and Kunlun He},
  title = {{Effective multi-modal clustering method via skip aggregation network for parallel scRNA-seq and scATAC-seq data}},
  journal = {Briefings in Bioinformatics},
  year = {2024},
  doi = {10.1093/bib/bbae102},
}
Figure