TL;DR
This paper introduces "False Discovery Rate Regression" (FDRR), a novel statistical method that leverages auxiliary information (covariates) to improve the power of multiple hypothesis testing. By allowing these covariates to influence the probability of a test being a true signal, FDRR significantly increases the number of discoveries in large-scale screening problems, as demonstrated in a neural synchrony detection task.
What Problem Does This Paper Address?
In many scientific fields, researchers conduct numerous statistical tests simultaneously (e.g., in genomics or neuroscience). A common approach to control for the increased chance of false positives is to use False Discovery Rate (FDR) control methods. However, these standard methods often treat all tests as exchangeable, ignoring potentially valuable auxiliary information about each test. This can lead to suboptimal results, especially when tests with similar characteristics tend to cluster in covariate space. For instance, in neuroscience, the likelihood of two neurons firing in synchrony might depend on their physical distance or functional similarity, information typically discarded by traditional FDR methods. This paper addresses the challenge of how to effectively incorporate such covariate information into the multiple testing framework to boost discovery power without compromising error control.
Methods
The core innovation is False Discovery Rate Regression (FDRR). It extends the popular two-groups model for FDR control by allowing the mixing fraction (the proportion of tests that are signals) to be a function of covariates.
- Two-Groups Model Foundation: The paper builds upon the established two-groups model, which assumes test statistics arise from a mixture of a null distribution ($f_0$) and an alternative distribution ($f_1$), with a mixing proportion ($c$).
- Covariate Integration: FDRR models the probability of a test being a signal as a function of covariates ($x_i$), denoted as $c(x_i)$. This is achieved through a logistic regression-like link function, $c(x_i) = G(s(x_i))$, where $s(x_i)$ is an unknown regression function.
- Empirical Bayes Approach: One key method involves estimating an "empirical null" ($f_0$) and then using predictive recursion to estimate the alternative distribution ($f_1$) and mixing proportion ($c$) without covariates. Subsequently, an Expectation-Maximization (EM) algorithm is used to estimate the regression coefficients ($\beta$) that link covariates to the signal probability.
- Full Bayes Approach: A fully Bayesian approach is also presented, which jointly estimates all model parameters, including the regression function and the mixing distribution, using Markov Chain Monte Carlo (MCMC) methods. This approach offers a natural way to quantify uncertainty.
- Predictive Recursion: A computationally efficient algorithm for estimating the mixing distribution in the two-groups model, particularly useful for its speed and theoretical guarantees.
Datasets
The primary dataset used for evaluation is from neural recordings of primary visual cortex (V1) in a rhesus macaque monkey. This dataset consists of simultaneous recordings from 128 neurons, generating thousands of pairs of neurons, each representing a null hypothesis of no interaction. The paper uses this dataset to demonstrate the application of FDRR to detect fine-time-scale neural interactions ("synchrony").
The paper also describes extensive simulation studies using various combinations of regression functions ($s(x)$) and prior distributions for the mixing parameter ($\pi(\theta)$) to validate the performance of FDRR. These simulations generated data sets with 10,000 test statistics each.
Key Insights and Ideas
- Covariates Matter: The most significant insight is that ignoring covariate information in multiple testing can lead to a substantial loss of power, especially when signals are not uniformly distributed across covariate space.
- FDRR as a Flexible Tool: FDRR offers a principled way to incorporate auxiliary information, moving beyond the assumption of exchangeable tests. This makes it applicable to a wider range of complex, high-dimensional data problems.
- Improved Power Without Compromising Error Control: The simulations demonstrate that FDRR can achieve higher true positive rates (power) for a fixed false discovery rate, particularly when covariate effects are present.
- Robustness to Absent Covariate Effects: When covariate effects are absent, FDRR remains robust and does not lead to inflated error rates, performing comparably to standard methods.
- Interpretability of Covariate Effects: The estimated regression functions in FDRR provide interpretable insights into how covariates influence the probability of a test being a true signal.
Key Results
- Neural Synchrony Detection: In the primary visual cortex neural data, FDRR detected approximately 50% more synchronous neuron pairs compared to a standard FDR-controlling analysis.
- Simulation Study Power Gains: In simulations where covariate effects were present, FDRR (both empirical Bayes and full Bayes versions) showed substantial power improvements, often between 40-50% in relative terms (or 8-12% in absolute terms) compared to standard methods like Benjamini-Hochberg.
- Error Rate Control: FDRR methods effectively controlled the false discovery rate at the nominal 10% level in simulations. While the empirical Bayes method showed a slight tendency to violate the nominal level when covariate effects were absent, the average FDR remained close to the target.
- Distance and Tuning Correlation Effects: In the neural data analysis, FDRR revealed that the probability of synchrony was strongly influenced by inter-neuron distance (closer neurons are more likely to be synchronous) and tuning-curve correlation (neurons with similar tuning curves are more likely to be synchronous). For example, a pair of neurons at the 25th percentile of distance had a significantly higher prior probability of synchrony (16.5%) compared to a pair at the 75th percentile (2.3%), holding tuning correlation constant.
Limitations and Future Work
- Computational Complexity: The full Bayes approach, while providing a natural way to quantify uncertainty, is computationally more intensive than the empirical Bayes approach.
- Empirical Null Estimation: Incorporating an empirical null in a fully Bayesian manner is challenging and often requires strong prior assumptions. The paper uses an empirical-Bayes approach for estimating the empirical null, which is then used in the full Bayes analysis.
- Model Specification: The paper assumes a logistic link and Gaussian error model, which could be modified for different problems. The choice of basis functions (B-splines) for modeling the regression functions also involves tuning.
- Future Work: The authors suggest that the FDRR framework is well-suited for situations where test statistics are not exchangeable, a common scenario in modern high-dimensional data analysis. The companion R package
FDRregimplements the described methods, facilitating further research and application. The paper also notes that while the empirical Bayes and full Bayes methods yielded similar results, the empirical Bayes method showed a slight tendency to violate nominal FDR levels when covariate effects were absent, an observation they did not fully understand and could be a direction for future investigation.
Summary updated