Out-of-distribution prediction with disentangled representations for single-cell RNA sequencing data

Mohammad Lotfollahi, Leander Dony, Harshita Agarwala, Fabian J. TheisbioRxiv 2021 · DOI

TL;DR

This paper introduces a method using a disentangled variational autoencoder (β-VAE) to learn interpretable representations of single-cell RNA sequencing (scRNA-seq) data. This approach enables accurate prediction of gene expression for cell types not seen during training, outperforming a state-of-the-art disentanglement method.

What Problem Does This Paper Address?

Single-cell RNA sequencing (scRNA-seq) generates vast amounts of data, and a key challenge is to extract meaningful biological insights from it. Specifically, researchers want to understand the underlying biological variations and create representations that are both informative and interpretable. This paper tackles the problem of "out-of-distribution" (OOD) prediction in scRNA-seq data. OOD prediction means predicting how cells will behave or express genes in conditions or cell types that were not present in the original training data. This is crucial for understanding biological processes like responses to perturbations or developmental trajectories where new cell states emerge. Existing methods often provide representations but lack interpretability or struggle with OOD scenarios.

Methods

The core of this work is the application of a β-Variational Autoencoder (β-VAE) for disentangled representation learning. A standard Variational Autoencoder (VAE) learns a compressed representation (latent space) of data. The β-VAE introduces a hyperparameter, β, which scales the KL divergence term in the VAE's objective function. This scaling encourages the model to learn more disentangled representations, meaning that individual dimensions in the latent space correspond to distinct, interpretable generative factors of the data (e.g., cell type, experimental condition).

The authors also compare their β-VAE approach to a dHSIC-VAE model. The dHSIC-VAE uses a Hilbert-Schmidt Independence Criterion (dHSIC) to enforce independence between latent dimensions, aiming for disentanglement. However, the authors note that dHSIC-VAE requires more hyperparameter tuning and doesn't inherently provide insights into the identity of learned dimensions.

Datasets

The paper evaluates its method on two primary datasets:

  • Peripheral Blood Mononuclear Cell (PBMC) scRNA-seq data (Kang et al., 2018): This dataset contains cells from both control and IFN-β stimulated conditions, with 12 different cell types. It was used to test the prediction of OOD "CD4 Naive T" cells after a perturbation.
  • Dentate Gyrus dataset (Hochgerner et al., 2018): This dataset captures developmental trajectories in the mouse brain, with multiple cell lineages. It was used to test the recovery of missing cell types from a developmental trajectory.

Key Insights and Ideas

  • Disentanglement for OOD Prediction: The paper strongly suggests that learning disentangled representations is a powerful strategy for enabling out-of-distribution predictions in scRNA-seq data. By isolating specific generative factors, the model can manipulate them independently to predict unseen states.
  • Interpretability is Key: The β-VAE's ability to disentangle factors like "cell type" and "condition" makes the learned representations interpretable. This interpretability is not just an academic exercise; it directly facilitates the OOD prediction task.
  • β-VAE Superiority: The β-VAE model consistently outperformed the dHSIC-VAE in both disentanglement quality and OOD prediction accuracy across the tested datasets. This suggests that the specific regularization strategy of β-VAE is well-suited for scRNA-seq data.
  • Developmental Trajectories as Disentangled Factors: The "cell type" dimension learned by the β-VAE also captured the order of development, acting as a proxy for a developmental trajectory. This highlights the potential of disentangled representations to uncover complex biological processes.

Key Results

  • Accurate OOD Prediction: The β-VAE model demonstrated accurate gene expression predictions for cell types absent from the training data in both the perturbation (PBMC) and developmental (Dentate Gyrus) datasets.
  • Improved Prediction Accuracy: In OOD prediction tasks, the β-VAE achieved more accurate predictions than the dHSIC-VAE. For instance, in the PBMC dataset, the predicted cells were closer to real cells in PCA space when using β-VAE.
  • Better Disentanglement: The β-VAE model achieved better disentanglement performance compared to dHSIC-VAE, as indicated by disentanglement scores in Table 1.
  • Quantitative Performance: The paper reports R² values for gene expression correlation between predicted and real cells, with values around 0.93-0.98 for the β-VAE model in OOD prediction tasks.

Limitations and Future Work

  • Hyperparameter Tuning: While the paper systematically searched for hyperparameters, the optimal settings for β-VAE and dHSIC-VAE might still be sensitive to the specific dataset and task.
  • Generalizability: The study focused on two specific datasets. Further validation on a wider range of scRNA-seq data types and biological contexts would be beneficial.
  • Future Directions: The authors envision that disentanglement learning on single-cell data can lead to more interpretable representations, ultimately improving our understanding of underlying biology and cellular heterogeneity. Future work could explore applying this approach to other complex biological questions and developing more sophisticated disentanglement techniques.

Summary updated

Abstract

Abstract Learning robust representations can help uncover underlying biological variation in scRNA-seq data. Disentangled representation learning is one approach to obtain such informative as well as interpretable representations. Here, we learn disentangled representations of scRNA-seq data using β variational autoencoder ( β -VAE) and apply the model for out-of-distribution (OOD) prediction. We demonstrate accurate gene expression predictions for cell-types absent from training in a perturbation and a developmental dataset. We further show that β -VAE outperforms a state-of-the-art disentanglement method for scRNA-seq in OOD prediction while achieving better disentanglement performance.
BibTeX
@misc{lotfollahi2021out,
  author = {Mohammad Lotfollahi and Leander Dony and Harshita Agarwala and Fabian J. Theis},
  title = {{Out-of-distribution prediction with disentangled representations for single-cell RNA sequencing data}},
  year = {2021},
  doi = {10.1101/2021.09.01.458535},
  howpublished = {bioRxiv},
}
Figure