Introduction
Large-scale multiple testing is a cornerstone of modern high-dimensional statistical inference. In fields such as genomics, neuroscience, and economics, researchers frequently test thousands or millions of hypotheses simultaneously. The traditional approach to managing the resulting false positives is the Benjamini-Hochberg (BH) procedure, which controls the False Discovery Rate (FDR). However, a significant limitation of the BH procedure and its variants is the assumption of "exchangeability"—the idea that every hypothesis is equally likely to be a discovery before the data is observed.
In practice, researchers often possess auxiliary information that suggests certain hypotheses are more promising than others. This information might come from previous studies, physical properties, or spatial relationships. While several methods have been developed to incorporate this data, they often require arbitrary "binning" or complex hyperparameter tuning. This paper introduces OrderShapeEM, a robust framework that adaptively incorporates auxiliary information to boost statistical power while maintaining strict FDR control.
The OrderShapeEM Framework
The proposed method is built upon a two-group mixture model. In this model, each test result (p-value) is assumed to originate from either a "null" distribution (representing no effect) or an "alternative" distribution (representing a real discovery). The key innovation lies in how the prior probability of a hypothesis being null ($\pi_{0i}$) is handled.
1. Varying Priors and Shape Constraints
Unlike standard models that assume a constant prior across all tests, OrderShapeEM allows $\pi_{0i}$ to vary for every individual hypothesis based on auxiliary information. To prevent overfitting and ensure biological plausibility, the authors impose a shape constraint. Specifically, they assume a monotone relationship: as the auxiliary relevance score increases, the probability of the hypothesis being null should decrease.
2. The OrderShapeEM Algorithm
To estimate these varying priors and the distribution of signals simultaneously, the authors developed a robust Expectation-Maximization (EM) algorithm. This algorithm integrates the Pool-Adjacent-Violators Algorithm (PAVA), a classic tool for isotonic regression. This approach makes the method "tuning-free," allowing it to scale efficiently to the massive datasets common in modern genomics without requiring manual intervention.
3. Optimal Rejection Rule
The framework employs a "step-up" procedure based on the Local False Discovery Rate (Lfdr). This rule is designed to maximize the expected number of true positives for a fixed FDR level, providing a theoretical guarantee of optimality under the model's assumptions.
Experimental Validation
Simulation Studies
The researchers conducted extensive simulations involving 10,000 hypotheses with varying signal densities (5% to 20%). The results demonstrated that OrderShapeEM consistently outperformed state-of-the-art competitors, including BH, Storey’s ST, AdaPT, and SABHA. The advantage was most pronounced when the signal was relatively weak but the auxiliary information was moderately informative.
Genome-Wide Association Studies (GWAS)
The methodology was applied to two large-scale datasets focused on Coronary Artery Disease (CAD):
- CARDIoGRAM: A meta-analysis of 14 studies (22,000 cases).
- C4D: A meta-analysis of 5 studies (15,000 cases).
By using one dataset as auxiliary information to inform the analysis of the other, OrderShapeEM identified 40 additional significant SNPs (genetic variants) that standard methods failed to detect. Crucially, many of these new discoveries were located in genes already known to be associated with cardiovascular and metabolic disorders, providing biological validation for the method's increased power.
Discussion and Key Insights
OrderShapeEM offers several advantages over existing "ordered" testing methods:
- Flexibility: While some methods stop rejecting hypotheses once they hit a certain point in a ranked list, OrderShapeEM allows for "out-of-order" rejections. A hypothesis with a very strong p-value can still be rejected even if its auxiliary rank was low.
- Robustness: By using isotonic regression (PAVA) rather than a rigid parametric formula, the model adapts to various types of auxiliary data without the risk of over-fitting.
- Scalability: The algorithm is computationally efficient and requires no hyperparameter tuning, making it suitable for "Big Data" applications.
Limitations and Future Work
The authors note that the method's performance is tied to the quality of the auxiliary information; if the information is purely noise, the method may be slightly less powerful than the standard BH procedure. Additionally, the current model assumes that null p-values follow a uniform distribution, which may not hold in cases of composite nulls or skewed data. Future research will explore extending this framework to handle more complex structures, such as spatial dependencies or hierarchical tree structures in microbiome data.
Summary updated