Dimensionality reduction and visualization of single-cell RNA-seq data with an improved deep variational autoencoder

Jing Jiang, Junlin Xu, Yuansheng Liu, Bosheng Song, Xiulan Guo, Xiangxiang Zeng, Quan ZouBriefings in Bioinformatics 2023 · DOI

Introduction

Single-cell RNA sequencing (scRNA-seq) has transformed biological research by providing high-resolution insights into gene expression at the individual cell level. Unlike traditional "bulk" sequencing, which averages signals across thousands of cells, scRNA-seq reveals the heterogeneity within complex tissues. However, the technology faces a significant technical hurdle: the "dropout" event. Due to the minute quantities of genetic material captured, many genes that are biologically active may not be detected, resulting in "zero" values in the data. This sparsity, combined with high dimensionality and technical noise, complicates downstream tasks like cell-type clustering and visualization.

The DREAM Architecture

To address these challenges, researchers developed DREAM (Dimensionality Reduction with an improved deep variational autoencoder). DREAM is a generative model designed to map high-dimensional scRNA-seq data into a low-dimensional latent space while accounting for the unique statistical properties of single-cell data.

The DREAM framework: An encoder-decoder architecture incorporating a Gaussian Mixture Model (GMM) latent space and a zero-inflated (ZI) layer to handle dropout events.

The model introduces three key innovations:

  • Gaussian Mixture Model (GMM) Latent Space: Unlike standard variational autoencoders (VAEs) that assume a single Gaussian distribution, DREAM uses a GMM prior. This acknowledges that biological samples are composed of distinct cell populations, allowing the model to naturally cluster data in the latent space.
  • Zero-Inflated (ZI) Layer: To mitigate the dropout problem, a specialized layer follows the decoder. It uses a probability model to distinguish between "true zeros" (biological absence) and "technical zeros" (dropouts), effectively denoising the data during the reconstruction process.
  • Differentiable Discrete Modeling: DREAM utilizes the "Gumbel-softmax trick" to make discrete dropout events differentiable, enabling the neural network to learn these parameters through standard backpropagation.

Experimental Results and Visualization

The performance of DREAM was evaluated against several established methods, including PCA, t-SNE, ZIFA, and VASC, across nine diverse datasets ranging from embryonic development to complex tissue samples.

Comparative visualization of five dimensionality reduction methods (PCA, t-SNE, ZIFA, VASC, and DREAM) across four benchmark datasets: Biase, Yan, Pollen, and Kolodziejczyk.

Superior Clustering and Separation

DREAM consistently outperformed competitors in clustering accuracy, as measured by the Adjusted Rand Index (ARI) and Normalized Mutual Information (NMI). A notable example is the Pollen dataset (Row C in the figure above), which contains 11 distinct cell types. While methods like t-SNE and VASC struggled to separate closely related populations such as Kera, 2338, IPS, and NPC, DREAM provided clear, distinct clusters for each.

Trajectory Recovery

Beyond static clustering, DREAM demonstrated high fidelity in capturing biological dynamics. In the Petropoulos dataset, which tracks human embryonic development, DREAM successfully reconstructed the continuous developmental trajectory from day E3 to E7. This suggests that the model preserves the underlying biological manifold better than traditional linear or standard deep learning approaches.

Discussion and Future Directions

DREAM represents a significant step forward in scRNA-seq analysis by integrating biological priors directly into the deep learning architecture. By modeling the dropout process rather than relying on pre-analysis imputation, it avoids the risk of introducing artificial patterns into the data.

However, the model's performance can be sensitive to hyperparameters such as the regularization dropout rate and training patience. Future iterations may focus on automated parameter optimization or extending the framework to "multi-omics" integration, combining transcriptomic data with epigenetic or proteomic signals to provide a more holistic view of cellular identity.

Source: Jiang, J., et al. (2023). Dimensionality reduction and visualization of single-cell RNA-seq data with an improved deep variational autoencoder. Briefings in Bioinformatics.

Summary updated

BibTeX
@article{jiang2023dimensionality,
  author = {Jing Jiang and Junlin Xu and Yuansheng Liu and Bosheng Song and Xiulan Guo and Xiangxiang Zeng and Quan Zou},
  title = {{Dimensionality reduction and visualization of single-cell RNA-seq data with an improved deep variational autoencoder}},
  journal = {Briefings in Bioinformatics},
  year = {2023},
  doi = {10.1093/bib/bbad152},
}
Figure