Trade-off between conservation of biological variation and batch effect removal in deep generative modeling for single-cell transcriptomics

Hui Li, Davis J. McCarthy, Heejung Shim, Susan Wei

TL;DR

This paper tackles the challenge of balancing two competing goals in single-cell RNA sequencing (scRNA-seq) data analysis: preserving biological variation and removing technical batch effects. They introduce a novel approach using Pareto Multi-Task Learning (Pareto MTL) to map this trade-off and propose Mutual Information Neural Estimation (MINE) as a superior method for quantifying batch effects compared to traditional techniques.

What Problem Does This Paper Address?

Single-cell RNA sequencing (scRNA-seq) generates high-dimensional data that requires dimensionality reduction to reveal underlying biological patterns. While models like scVI are good at learning these patterns, they often struggle to explicitly control for "batch effects" – technical variations introduced by different experimental conditions, labs, or time points. These batch effects can obscure true biological signals. The core problem is that aggressively removing batch effects can inadvertently remove genuine biological variation, and vice versa. Researchers need a way to understand and navigate this inherent trade-off to make informed decisions about their data analysis.

Methods

The authors build upon the scVI deep generative model. Their primary contribution is the application of Pareto Multi-Task Learning (Pareto MTL) to estimate the Pareto front. This technique allows them to explore the entire spectrum of trade-offs between two objectives:

  1. Conservation of biological variation (Un): Measured by the generative loss of scVI.
  2. Removal of batch effects (Vn): Quantified using a novel approach.

Instead of a simple weighted sum (scalarization), Pareto MTL aims to find a set of "Pareto optimal" solutions that represent the best possible compromises.

For measuring batch effects (Vn), they propose using Mutual Information Neural Estimation (MINE). MINE leverages deep neural networks to estimate the mutual information between the latent representation of the data and the batch labels. This is presented as an improvement over the more standard Maximum Mean Discrepancy (MMD), which requires careful kernel selection and bandwidth tuning.

Datasets

The study utilizes two scRNA-seq datasets:

  • Tabula Muris (TM-MARROW): An integration of two mouse bone marrow datasets.
  • MACAQUE-RETINA: Raw macaque retinal single-cell count data, focusing on bipolar cells.

These datasets are used for training and evaluating the proposed methods.

Key Insights and Ideas

  • The Pareto Front as a Decision-Making Tool: Visualizing the Pareto front provides researchers with a comprehensive view of the trade-offs, enabling them to select a solution that best suits their specific research question (e.g., prioritizing batch removal over biological variation, or vice versa).
  • Pareto MTL Outperforms Scalarization: The Pareto MTL approach generates a more diverse and well-distributed set of Pareto candidates compared to traditional scalarization methods, which tend to cluster at the extremes of the trade-off curve.
  • MINE as a Superior Batch Effect Metric: MINE offers a more robust and less parameter-sensitive way to measure batch effects compared to MMD, leading to more interpretable trade-off curves that better respect subproblem ordering.
  • Biological Interpretation of Trade-offs: The authors demonstrate how different points on the Pareto front correspond to varying levels of batch effect removal and biological variation, which can be visually confirmed using t-SNE plots and quantitatively assessed with metrics like batch entropy.

Key Results

  • Pareto MTL, when combined with MINE, successfully estimates a well-distributed Pareto front for the trade-off between biological variation conservation and batch effect removal.
  • The proposed MINE metric for batch effect estimation shows benefits over MMD, particularly in producing trade-off curves that respect subproblem ordering and are interpretable.
  • Quantitative metrics (percentage of non-dominated points, hypervolume, NDC) show that Pareto MTL generally leads to better Pareto front estimations than scalarization, especially when using MINE. For instance, Pareto MTL with MINE achieved higher percentages of non-dominated points and better NDC values.
  • The study provides concrete examples (Fig. 2B and 2C) showing how different Pareto candidates result in distinct levels of batch mixing and cell type clustering.

Limitations and Future Work

  • Computational Cost of MINE: While MINE is presented as superior, it is computationally more demanding than MMD.
  • Adversarial Training Instability: Using MINE in conjunction with scVI requires adversarial training, which can be unstable for inexperienced users.
  • MINE Architecture Design: Designing the optimal neural network architecture for MINE can be challenging, although the authors found their results to be robust to architectural choices in their analysis.
  • Future Work: The authors suggest incorporating other deep generative models and generalizing the approach to other nuisance factors beyond batch effects, such as quality control metrics. They also propose exploring other surrogate metrics for evaluating trade-offs.

Summary updated

Abstract

Abstract Single-cell RNA sequencing (scRNA-seq) technology has contributed significantly to diverse research areas in biology, from cancer to development. Since scRNA-seq data is high-dimensional, a common strategy is to learn low-dimensional latent representations better to understand overall structure in the data. In this work, we build upon scVI, a powerful deep generative model which can learn biologically meaningful latent representations, but which has limited explicit control of batch effects. Rather than prioritizing batch effect removal over conservation of biological variation, or vice versa, our goal is to provide a bird’s eye view of the trade-offs between these two conflicting objectives. Specifically, using the well established concept of Pareto front from economics and engineering, we seek to learn the entire trade-off curve between conservation of biological variation and removal of batch effects. A multi-objective optimisation technique known as Pareto multi-task learning (Pareto MTL) is used to obtain the Pareto front between conservation of biological variation and batch effect removal. Our results indicate Pareto MTL can obtain a better Pareto front than the naive scalarization approach typically encountered in the literature. In addition, we propose to measure batch effect by applying a neural-network based estimator called Mutual Information Neural Estimation (MINE) and show benefits over the more standard Maximum Mean Discrepancy (MMD) measure. The Pareto front between conservation of biological variation and batch effect removal is a valuable tool for researchers in computational biology. Our results demonstrate the efficacy of applying Pareto MTL to estimate the Pareto front in conjunction with applying MINE to measure the batch effect.
BibTeX
@article{li2022trade,
  author = {Hui Li and Davis J. McCarthy and Heejung Shim and Susan Wei},
  title = {{Trade-off between conservation of biological variation and batch effect removal in deep generative modeling for single-cell transcriptomics}},
  journal = {BMC Bioinformatics},
  year = {2022},
  doi = {10.1186/s12859-022-05003-3},
}
Figure