Trade-off between conservation of biological variation and batch effect removal in deep generative modeling for single-cell transcriptomics

Hui Li, Davis J. McCarthy, Heejung Shim, Susan Wei

TL;DR

This paper tackles the challenge of balancing the preservation of biological signals with the removal of technical noise (batch effects) in single-cell RNA sequencing data. They introduce a novel approach using Pareto multi-task learning (Pareto MTL) and a new batch effect measurement technique (MINE) to visualize and navigate this trade-off, offering researchers a more nuanced understanding than traditional methods.

What Problem Does This Paper Address?

Single-cell RNA sequencing (scRNA-seq) is a powerful tool for understanding cellular heterogeneity. However, scRNA-seq data is notoriously noisy, with "batch effects" – variations arising from experimental conditions rather than true biological differences – being a major concern. Deep generative models like scVI are excellent at learning low-dimensional representations of this data, capturing biological variation. Yet, they often struggle to explicitly disentangle biological signals from batch effects. Existing methods tend to either prioritize batch effect removal at the expense of biological information or vice-versa. This paper aims to address this by providing a comprehensive view of the trade-off between these two competing objectives, allowing researchers to make informed decisions about how to best process their data.

Methods

The core of this work lies in framing the problem as a multi-objective optimization task: minimizing both the generative loss (representing biological variation) and a batch effect measure.

  • scVI as the Foundation: The authors build upon the scVI (single-cell variational inference) framework, a deep generative model that uses a zero-inflated negative binomial (ZINB) distribution to model scRNA-seq data and learns low-dimensional latent representations.
  • Pareto Front Concept: They borrow the concept of a "Pareto front" from economics and engineering. A Pareto front represents a set of optimal solutions where no single objective can be improved without degrading another. In this context, it visualizes the spectrum of solutions balancing biological variation and batch effect removal.
  • Pareto Multi-Task Learning (Pareto MTL): To estimate this Pareto front, they employ Pareto MTL, a sophisticated technique that aims to find a well-distributed set of Pareto optimal points. This is contrasted with the "naive scalarization" approach, which typically involves sweeping a regularization parameter (λ) and often struggles to capture the full range of trade-offs, especially in non-convex regions of the Pareto front.
  • Mutual Information Neural Estimator (MINE) for Batch Effect Measurement: A key novel contribution is the use of MINE to quantify batch effects. MINE, a deep neural network-based method, estimates the mutual information between the learned latent representations (z) and the batch labels (s). Higher mutual information indicates stronger dependence on batch, thus a larger batch effect. This is presented as an improvement over the more standard Maximum Mean Discrepancy (MMD) method, which requires careful kernel bandwidth selection.

Datasets

The proposed methods were evaluated on two publicly available single-cell RNA sequencing datasets:

  • TM-MARROW: An integrated dataset from the Tabula Muris project, combining two bone marrow datasets (MarrowTM-10x and MarrowTM-ss2).
  • MACAQUE-RETINA: A dataset containing raw macaque retinal cell count data, focusing on bipolar cells from foveal and peripheral regions.

Key Insights and Ideas

  • Visualizing the Trade-off is Crucial: The paper strongly advocates for understanding the entire Pareto front rather than settling for a single "optimal" point determined by a heuristic regularization parameter. This provides a richer picture of the available solutions.
  • Pareto MTL Outperforms Scalarization: Pareto MTL consistently produces a more diverse and well-distributed set of Pareto optimal points compared to the naive scalarization approach, which tends to cluster solutions at the extremes.
  • MINE is a Promising Batch Effect Metric: MINE proves to be a robust and interpretable measure of batch effect, producing trade-off curves that respect candidate ordering and are more amenable to surrogate metric analysis than MMD.
  • No Single "Best" Solution: The authors emphasize that all points on the Pareto front are equally optimal in a multi-objective sense. The choice of a specific point depends on the researcher's priorities regarding biological variation versus batch effect removal.

Key Results

  • Well-Distributed Pareto Fronts: Pareto MTL, especially when combined with MINE, successfully estimated well-distributed Pareto fronts on both datasets, visualized through plots of generative loss (Un) versus batch effect (Vn).
  • MINE vs. MMD: MINE generally led to better-behaved Pareto fronts, with trade-off points that respected candidate ordering and showed clearer trends in surrogate metric spaces compared to MMD.
  • Quantitative Improvements: Table 1 shows that Pareto MTL generally achieved higher percentages of non-dominated points and higher NDC (number of distinct choices), indicating better diversity and coverage of the Pareto front compared to scalarization.
  • Superior Clustering: The paper demonstrates that Pareto MTL with MINE can achieve better cell type clustering (as measured by metrics like ASW, ARI, NMI, UCA) while simultaneously reducing batch effects, outperforming scVI alone in some cases.

Limitations and Future Work

  • Computational Cost of MINE: While effective, MINE is computationally more demanding than MMD and requires adversarial training, which can be unstable.
  • MINE Architecture Design: Designing the optimal neural network architecture for MINE can be challenging, although the authors found their chosen architecture to be robust.
  • Generalizability to Other Architectures: The current work focuses on scVI. Future work could explore incorporating other deep architectures (e.g., SAUCIE, DESC) into the Pareto MTL framework.
  • Incorporating Other Nuisance Factors: The authors suggest extending the framework to account for other quality control metrics beyond batch effects, such as alignment errors.

Summary updated

Abstract

Background Single-cell RNA sequencing (scRNA-seq) technology has contributed significantly to diverse research areas in biology, from cancer to development. Since scRNA-seq data is high-dimensional, a common strategy is to learn low-dimensional latent representations better to understand overall structure in the data. In this work, we build upon scVI, a powerful deep generative model which can learn biologically meaningful latent representations, but which has limited explicit control of batch effects. Rather than prioritizing batch effect removal over conservation of biological variation, or vice versa, our goal is to provide a bird’s eye view of the trade-offs between these two conflicting objectives. Specifically, using the well established concept of Pareto front from economics and engineering, we seek to learn the entire trade-off curve between conservation of biological variation and removal of batch effects. Results A multi-objective optimisation technique known as Pareto multi-task learning (Pareto MTL) is used to obtain the Pareto front between conservation of biological variation and batch effect removal. Our results indicate Pareto MTL can obtain a better Pareto front than the naive scalarization approach typically encountered in the literature. In addition, we propose to measure batch effect by applying a neural-network based estimator called Mutual Information Neural Estimation (MINE) and show benefits over the more standard maximum mean discrepancy measure. Conclusion The Pareto front between conservation of biological variation and batch effect removal is a valuable tool for researchers in computational biology. Our results demonstrate the efficacy of applying Pareto MTL to estimate the Pareto front in conjunction with applying MINE to measure the batch effect.
BibTeX
@article{li2022trade,
  author = {Hui Li and Davis J. McCarthy and Heejung Shim and Susan Wei},
  title = {{Trade-off between conservation of biological variation and batch effect removal in deep generative modeling for single-cell transcriptomics}},
  journal = {BMC Bioinformatics},
  year = {2022},
  doi = {10.1186/s12859-022-05003-3},
}
Figure