dropClust2 : An R package for resource efficient analysis of large scale single cell RNA-Seq data

Debajyoti Sinha, Pradyumn Sinha, Ritwik Saha, Sanghamitra Bandyopadhyay, Debarka SenguptabioRxiv 2019 · DOI

TL;DR

This paper introduces dropClust2, a significantly improved R package for analyzing large single-cell RNA sequencing (scRNA-seq) datasets. It addresses the computational bottlenecks and usability issues of its predecessor by offering a faster, more memory-efficient, and entirely R-based solution, complete with a novel batch effect removal algorithm for integrated analysis.

What Problem Does This Paper Address?

Single-cell RNA sequencing (scRNA-seq) technologies have revolutionized our ability to understand cellular heterogeneity, enabling the identification of rare cell types. However, the sheer volume of data generated by these technologies (often tens of thousands of cells) poses significant computational challenges for analysis. Existing tools, like the original dropClust, while effective, suffered from cumbersome multi-language dependencies (R, Python, C) and high resource requirements, making them difficult to install and run on standard hardware. This paper tackles the need for a more accessible, efficient, and robust tool for clustering and analyzing large-scale scRNA-seq data, particularly for integrating datasets that may have been generated under different experimental conditions.

Methods

dropClust2 builds upon the original dropClust algorithm, which uses Locality Sensitive Hashing (LSH) and Structure Preserving Sampling (SPS) for efficient clustering. The key improvements in dropClust2 include:

  • Pure R Implementation: The entire package is now written in R, eliminating the need for complex multi-language installations and improving portability.
  • Faster Principal Component Selection: Instead of the slow Gaussian Mixture Models (GMM) used previously, dropClust2 employs Gaussian Kernel Density Estimators and cubic splines to identify principal components, leading to a significant speed-up.
  • Novel Batch Effect Removal: A new algorithm is introduced that leverages the robustness of gene expression ranks. It identifies and randomly discards one gene from pairs with similar expression distributions that inconsistently flip ranks across datasets, thereby mitigating batch effects for integrated analysis.
  • Improved Cluster Assignment: A user-defined parameter conf is introduced to refine the post-hoc cluster assignment, allowing for a more robust assignment when nearest neighbor frequencies are not clearly dominant.
  • Web-based Interface: A user-friendly Shiny web application is provided, enabling online analysis of large scRNA-seq datasets without requiring local installation.

Datasets

The paper evaluates dropClust2 using the PBMC 68K dataset. Additionally, it mentions the use of a cohort of ~6K PBMC transcriptomes from two independent experiments to demonstrate the advantage of batch effect removal.

Key Insights and Ideas

  • Democratizing scRNA-seq Analysis: By creating a pure R package and a web server, dropClust2 makes advanced scRNA-seq analysis more accessible to a wider range of researchers, not just those with extensive computational expertise or high-performance computing resources.
  • Efficiency Through Algorithmic Refinement: The shift from GMM to density estimation for principal component selection highlights how algorithmic improvements can dramatically reduce computational time and resource usage without sacrificing analytical quality.
  • The Power of Rank-Based Methods for Batch Correction: The novel batch effect removal strategy underscores the idea that focusing on relative expression (ranks) can be more robust to technical variations between datasets than absolute expression values, a crucial insight for integrative analyses.
  • Balancing Automation and Control: The introduction of the conf parameter for cluster assignment demonstrates a thoughtful approach to balancing automated clustering with user control, allowing for more nuanced interpretation of results.

Key Results

dropClust2 demonstrates substantial performance improvements over its predecessor:

  • Speed: Processing a 68K PBMC dataset, dropClust2 takes less than 3 minutes, compared to 7 minutes for the original dropClust.
  • Memory: Memory usage is drastically reduced from 20GB to 3.5GB for the same dataset.
  • Accuracy: On the PBMC 68K dataset, dropClust2 achieved an Adjusted Rand Index (ARI) of 0.4, outperforming the former implementation (0.37) and significantly outperforming Seurat and SCANPY (as shown in Figure 1(c)).
  • Batch Effect Removal: Supplementary Figure S3 shows the advantage of the batch effect removal on a ~6K PBMC dataset from two experiments.

Limitations and Future Work

  • Limitations: The paper does not explicitly detail limitations of dropClust2 itself, but the performance comparison in Figure 1(c)-(e) shows that while dropClust2 excels in ARI, time, and memory, there might be specific scenarios where other methods perform differently. The ARI of 0.4 suggests that clustering accuracy can still be improved.
  • Future Work: The paper focuses on the development and validation of dropClust2. Future work could involve further refinement of the batch effect removal algorithm, exploration of different clustering strategies, and expanding the range of scRNA-seq data types it can effectively handle.

Summary updated

Abstract

ABSTRACT DropClust leverages Locality Sensitive Hashing (LSH) to speed up clustering of large scale single cell expression data. It makes ingenious use of structure persevering sampling and modality based principal component selection to rescue minor cell types. Existing implementation of dropClust involves interfacing with multiple programming languages viz . R, python and C, hindering seamless installation and portability. Here we present dropClust2, a complete R package that’s not only fast but also minimally resource intensive. DropClust2 features a novel batch effect removal algorithm that allows integrative analysis of single cell RNA-seq (scRNA-seq) datasets. Availability and implementation dropClust2 is freely available at https://debsinha.shinyapps.io/dropClust/ as an online web service and at https://github.com/debsin/dropClust as an R package.
BibTeX
@misc{sinha2019i,
  author = {Debajyoti Sinha and Pradyumn Sinha and Ritwik Saha and Sanghamitra Bandyopadhyay and Debarka Sengupta},
  title = {{<i>dropClust2</i> : An R package for resource efficient analysis of large scale single cell RNA-Seq data}},
  year = {2019},
  doi = {10.1101/596924},
  howpublished = {bioRxiv},
}
Figure