Authors: David Donoho and Jiashun Jin
Abstract
We apply False Discovery Rate (FDR) thresholding to a non-Gaussian vector whose coordinates $X_i, i=1,..., n$, are independent exponential with individual means $\mu_i$. The vector $\mu =(\mu_i)$ is thought to be sparse, with most coordinates 1 but a small fraction significantly larger than 1; roughly, most coordinates are simply 'noise,' but a small fraction contain 'signal.' We measure risk by per-coordinate mean-squared error in recovering $\log(\mu_i)$, and study minimax estimation over parameter spaces defined by constraints on the per-coordinate p-norm of $\log(\mu_i)$: $\frac{1}{n}\sum_{i=1}^n\log^p(\mu_i)\leq \eta^p$. We show for large $n$ and small $\eta$ that FDR thresholding can be nearly Minimax. The FDR control parameter $0 < q < 1$ plays an important role: when $q \leq 1/2$, the FDR estimator is nearly minimax, while choosing a fixed $q > 1/2$ prevents near minimaxity. These conclusions mirror those found in the Gaussian case. The techniques developed here seem applicable to a wide range of other distributional assumptions, other loss measures and non-i.i.d. dependency structures.
1. Introduction
In high-dimensional statistics, researchers frequently encounter the "needle in a haystack" problem: identifying a small number of significant signals (the needles) buried within a massive volume of noise (the haystack). While the Gaussian distribution is the standard model for noise in many contexts, many real-world phenomena—such as failure times in reliability engineering, power levels in signal processing, or the distribution of p-values—are better characterized by exponential distributions.
This paper, authored by David Donoho and Jiashun Jin, investigates whether the False Discovery Rate (FDR) thresholding procedure—originally designed for multiple hypothesis testing—can serve as an optimal estimation tool for sparse exponential data. Specifically, the authors explore the "minimax" properties of FDR thresholding, asking if it can achieve the best possible performance in the worst-case scenario as the data becomes increasingly sparse.
2. Problem Formulation: Sparse Exponential Signals
Consider a vector of $n$ independent observations $X_i$, where each $X_i$ follows an exponential distribution with mean $\mu_i$. The vector $\mu = (\mu_i)$ is assumed to be sparse, meaning that for a large majority of indices, $\mu_i = 1$ (representing pure noise). A small fraction $\eta$ of the coordinates have $\mu_i > 1$ (representing signal).
The goal is to recover the values of $\log(\mu_i)$. The authors utilize the Mean Squared Error (MSE) on the log-scale as the risk metric. This choice is mathematically motivated: because exponential data is inherently multiplicative, the log-scale provides a natural framework for measuring estimation error. The parameter space is defined by the $p$-norm constraint: $$\frac{1}{n}\sum_{i=1}^n\log^p(\mu_i)\leq \eta^p$$
3. Methodology: FDR Thresholding and the Functional Approach
The False Discovery Rate (FDR) procedure, introduced by Benjamini and Hochberg, is a data-dependent thresholding rule. Unlike fixed thresholds, FDR thresholding sorts the observations and selects a cutoff based on a control parameter $q$.
To analyze this procedure theoretically, Donoho and Jin introduce the FDR functional ($T_q$). This mathematical abstraction treats the data-dependent threshold as a function of the underlying probability distribution. This allows the authors to prove consistency and convergence rates as $n \to \infty$. Furthermore, they employ a Minimax-Bayes Bridge: they first solve a Bayesian version of the problem using a "least favorable" prior and then demonstrate that the frequentist minimax risk converges to this Bayesian risk in the limit of extreme sparsity.
4. Key Results: The $q \leq 1/2$ Rule
The paper’s most significant contribution is the discovery of a phase transition in the performance of the FDR estimator based on the parameter $q$:
- Asymptotic Minimaxity ($q \leq 1/2$): When the FDR control parameter is set to $1/2$ or less, the ratio of the FDR estimator's risk to the ideal minimax risk converges to 1 as $n \to \infty$ and $\eta \to 0$. This means the procedure is nearly optimal.
- Suboptimality ($q > 1/2$): If the researcher is "too greedy" and sets $q > 1/2$, the estimator fails to be minimax. In this regime, the risk ratio converges to $q/(1-q)$, which is strictly greater than 1.
This result provides a beautiful interpretation of FDR: by ensuring that there are at least as many true discoveries as false positives (the condition met when $q \leq 1/2$), the threshold naturally aligns with the optimal cutoff for minimizing total estimation error.
5. Discussion and Implications
The findings in this paper mirror previous results found in Gaussian settings, suggesting that the effectiveness of FDR thresholding is not a fluke of the Normal distribution. Instead, it appears to be a robust, "all-weather" tool for sparse estimation across different distributional families.
The authors note that the optimal threshold $t_0$ scales logarithmically with the sparsity ($t_0 \approx p \log(1/\eta)$). This provides practitioners with a clear rule of thumb for threshold selection in sparse environments.
6. Limitations and Future Work
While the results are robust, they rely on the Monotone Likelihood Ratio (MLR) property of the exponential distribution. Distributions lacking this property might exhibit different behavior. Additionally, the study assumes independent observations. While the authors suggest that their "FDR functional" approach could be extended to handle weakly dependent data, formal proofs for such cases—and for other distributions like the Poisson distribution—remain subjects for future research.
7. Conclusion
Donoho and Jin demonstrate that FDR thresholding is more than a heuristic for p-values; it is a mathematically rigorous method for achieving minimax optimality in sparse, non-Gaussian estimation. For practitioners, the takeaway is clear: when dealing with sparse signals in exponential noise, setting $q \leq 1/2$ ensures that the FDR procedure performs as well as the best possible theoretical estimator.
Summary updated