Distribution-free complex hypothesis testing for single-cell RNA-seq differential expression analysis

Marine Gauthier, Denis Agniel, Rodolphe Thiébaut, Hejblum Boris P.Zenodo 2021 · DOI

TL;DR

This paper introduces ccdf, a novel, distribution-free method for differential gene expression analysis in single-cell RNA sequencing (scRNA-seq) data. ccdf can handle complex experimental designs with multiple conditions and covariates, overcoming limitations of existing tools that often rely on strong assumptions and are restricted to two-group comparisons.

What Problem Does This Paper Address?

Single-cell RNA sequencing (scRNA-seq) generates high-resolution gene expression data, enabling deeper biological insights. However, analyzing this data for differential gene expression (DE) presents challenges. Many current methods make strong assumptions about the underlying data distribution, which are hard to verify in practice and can lead to inaccurate results. Furthermore, as scRNA-seq studies become more complex, involving multiple conditions, continuous variables, or the need to account for confounding factors (covariates), existing tools often fall short. Most methods are limited to comparing just two conditions, hindering their applicability to more intricate biological questions. This paper addresses the need for a flexible, robust, and distribution-free method that can handle complex experimental designs and covariates in scRNA-seq DE analysis.

Methods

The core of the ccdf method is a conditional independence test (CIT) based on conditional cumulative distribution functions (CCDFs). Instead of assuming specific data distributions, ccdf estimates CCDFs using multiple regressions. This approach allows it to test the association between gene expression (Y) and one or more variables of interest (X), while potentially adjusting for covariates (Z).

Key methodological aspects include:

  • Distribution-Free Approach: ccdf does not rely on parametric assumptions about the gene expression distribution, making it more robust to real-world scRNA-seq data characteristics, including zero inflation.
  • Conditional Independence Test (CIT): The method tests the null hypothesis of conditional independence between gene expression (Y) and the variable(s) of interest (X) given covariates (Z), denoted as $H_0: Y \perp X | Z$.
  • CCDF Estimation: CCDFs are estimated using a regression technique, specifically by modeling the probability $P(Y < w_j | X=x, Z=z)$ for a set of thresholds $w_j$.
  • Asymptotic Test: The paper derives the asymptotic distribution of the ccdf test statistic, allowing for efficient computation with large sample sizes.
  • Permutation Test: For smaller sample sizes where asymptotic assumptions may not hold, a permutation test is provided to ensure accurate control of Type I error.
  • Flexibility: ccdf can handle both discrete and continuous variables of interest and multiple covariates.

Datasets

The paper utilizes the following datasets for evaluation:

  • Simulated Data: Extensive simulations were performed using data generated from negative binomial distributions and mixtures of negative binomial distributions to mimic scRNA-seq count data. These simulations covered various scenarios, including:
  • Two-condition comparisons with different types of differential expression (DE) patterns (difference in mean, modality, proportion, or both).
  • Multiple condition comparisons (four groups).
  • Two-condition comparisons with a confounding covariate.
  • Real Data Benchmarks:
    • Positive Control Dataset: Data from Islam et al. (2011) was used, focusing on 1,000 validated genes to assess true positive rates.
  • Negative Control Dataset: Data from Grün et al. (2014) was used to evaluate false positive rates.
  • COVID-19 Patient Data: A large scRNA-seq dataset from Kusnadi et al. (2021) of 84,140 virus-reactive CD8+ T cells from COVID-19 patients was analyzed to demonstrate ccdf's application to complex real-world data with multiple severity groups and cell subpopulations.

Key Insights and Ideas

  • The "Curse of Dimensionality" in Distribution Estimation: Estimating probability density functions becomes computationally challenging and requires careful bandwidth selection as the number of variables increases. CCDFs, on the other hand, are more amenable to estimation in high-dimensional settings without requiring bandwidth tuning.
  • The Importance of Conditioning: The analysis of the COVID-19 dataset highlights how ignoring confounding variables (like cell clusters) can lead to an inflated number of differentially expressed genes and potentially misleading biological interpretations. ccdf's ability to adjust for covariates is crucial for accurate DE analysis.
  • Distribution-Free is Key for scRNA-seq: The inherent complexity and zero-inflation in scRNA-seq data make distribution-free methods like ccdf particularly well-suited, avoiding the pitfalls of relying on potentially incorrect parametric assumptions.
  • Versatility Beyond Two Groups: The paper demonstrates that ccdf can effectively handle experimental designs with more than two conditions, a significant advantage over many existing DE analysis tools.

Key Results

  • Simulation Performance:
    • ccdf showed good statistical performance across various complex simulation scenarios, including multiple conditions.
  • In two-condition benchmarks, ccdf retained competitive performance with state-of-the-art methods like MAST, scDD, and SigEMD, particularly in detecting differential expression in mean (DE) and modality (DM).
  • ccdf demonstrated excellent control of Type-I error and False Discovery Rate (FDR) across different sample sizes and scenarios.
  • The permutation test for ccdf showed higher power than the asymptotic test for small sample sizes.
  • Real Data Benchmarks:
    • On the positive control dataset, ccdf achieved the highest true positive rate (0.734) and identified more genes in common with the gold standard compared to SigEMD, scDD, and MAST.
  • On the negative control dataset, ccdf and MAST correctly identified zero DE genes, while scDD and SigEMD reported false positives.
  • COVID-19 Application:
    • Analyzing the COVID-19 dataset, ccdf identified 6,290 DE genes when adjusting for cell clusters, compared to 8,181 DE genes without adjustment. This difference underscores the impact of confounding factors.
  • The analysis revealed that adjusting for cell clusters reduced the significance of some genes in the Interferon (IFN) pathway, suggesting that differences in gene abundance were partly due to cell population composition rather than solely disease severity.

Limitations and Future Work

  • Computational Cost of Permutation Test: While the asymptotic test is fast, the permutation test can be time-consuming for very large datasets, although parallelization can mitigate this.
  • Multi-Sample Data: Currently, ccdf cannot directly analyze multi-sample data, such as biological replicates or repeated measurements. The authors suggest modifying the working model to accommodate this.
  • Choice of Thresholds: The number of thresholds used for CCDF estimation impacts computation time and statistical power. While ccdf uses evenly spaced thresholds by default, manual selection based on prior knowledge is an option.
  • Alternative Parameterizations: While logistic regression could be a more "natural" choice for modeling binary outcomes, ccdf currently uses linear regression for simplicity and speed. Exploring other parameterizations is a potential future direction.
  • Other Test Statistics: The paper focuses on a specific test statistic derived from the estimated coefficients. Investigating other test statistics based on the same CIT framework could be beneficial.

Summary updated

Abstract

State-of-the-art methods for single-cell RNA sequencing (scRNA-seq) Differential Expression Analysis (DEA) often rely on strong distributional assumptions that are difficult to verify in practice. Furthermore, while the increasing complexity of clinical and biological single-cell studies calls for greater tool versatility, the majority of existing methods only tackle the comparison between two conditions. We propose a novel, distribution-free, and flexible approach to DEA for single-cell RNA-seq data. This new method, called ccdf, tests the association of each gene expression with one or many variables of interest (that can be either continuous or discrete), while potentially adjusting for additional covariates. To test such complex hypotheses, ccdf uses a conditional independence test relying on the conditional cumulative distribution function, estimated through multiple regressions. We provide the asymptotic distribution of the ccdf test statistic as well as a permutation test (when the number of observed cells is not sufficiently large). ccdf substantially expands the possibilities for scRNA-seq DEA studies: it obtains good statistical performance in various simulation scenarios considering complex experimental designs (i.e. beyond the two condition comparison), while retaining competitive performance with state-of-the-art methods in a two-condition benchmark. We apply \texttt{ccdf} to a large publicly available scRNA-seq dataset of 84,140 SARS-CoV-2 reactive CD8+ T cells, in order to identify the diffentially expressed genes across 3 groups of COVID-19 severity (mild, hospitalized, and ICU) while accounting for seven different cellular subpopulations.
BibTeX
@inproceedings{gauthier2021distribution,
  author = {Marine Gauthier and Denis Agniel and Rodolphe Thiébaut and HEJBLUM Boris P.},
  title = {{Distribution-free complex hypothesis testing for single-cell RNA-seq differential expression analysis}},
  booktitle = {Zenodo},
  year = {2021},
  doi = {10.5281/zenodo.5701445},
}

Loading related content...

Figure