Weighted mining of massive collections of $p$-values by convex\n optimization

Edgar Dobriban

TL;DR

This paper introduces "Princessp," a novel method for analyzing massive collections of p-values using convex optimization. It allows researchers to prioritize certain hypotheses by assigning them higher weights and to impose practical constraints like monotonicity, overcoming limitations of existing methods and scaling efficiently to millions of tests.

What Problem Does This Paper Address?

In fields like genomics and cosmology, researchers often deal with vast numbers of p-values from statistical tests, aiming to identify significant "hits" while controlling error rates (like the False Discovery Rate or Family-Wise Error Rate). A key challenge is that traditional methods treat all hypotheses equally, failing to leverage prior knowledge that might suggest certain hypotheses are more likely to be true or have larger effect sizes. While weighted p-value methods exist, they often struggle with practical constraints such as ensuring weights are always positive or smoothly increasing with effect size, and they don't scale well to the massive datasets now common.

Methods

The core of Princessp is a constrained convex optimization framework. The authors formulate the problem of finding optimal p-value weights to maximize statistical power as a convex optimization problem. This is achieved by leveraging the concavity of Receiver Operating Characteristic (ROC) curves, particularly for monotone likelihood ratio families and two-sided tests in exponential families. Princessp allows for the incorporation of various "practitioner-friendly" constraints, including:

  • Monotonicity: Weights increase with hypothesized effect size.
  • Boundedness: Weights are kept within a specified lower and upper range.
  • Stratification: Hypotheses within predefined groups receive equal weights.
  • Smoothness: Weights vary smoothly across hypotheses, especially useful for spatially structured data.

For computational efficiency, especially with millions of hypotheses, they develop an interior-point method combined with a novel subsampling strategy to avoid ill-conditioning issues that arise with naive applications.

Datasets

The paper uses several standard genomics datasets for genome-wide association studies (GWAS) to evaluate Princessp. These include:

  • CARDIOGRAM and C4D (coronary artery disease)
  • Blood lipids study
  • Schizophrenia study
  • Chronic Kidney Disease Consortium (eGFR creatinine)
  • 90Plus dataset (aging)

These datasets consist of hundreds of thousands to millions of single nucleotide polymorphisms (SNPs) with associated p-values.

Key Insights and Ideas

  • Convexity is Key: By reformulating the optimal weighting problem in terms of convex optimization, the authors unlock the ability to handle complex constraints that were previously intractable.
  • Practitioner-Friendly Constraints: The ability to enforce monotonicity and boundedness on weights is crucial for real-world applicability. This addresses a major drawback of earlier optimal weighting methods that could produce unstable or counter-intuitive weights.
  • Bounded Weights Improve Performance: Empirically, imposing a lower bound on weights (e.g., $w_i \ge 0.5$) can significantly improve the performance of p-value weighting methods, preventing strong p-values from being unduly diminished by unreliable prior information.
  • Scalability: The development of an efficient interior-point method with subsampling allows Princessp to handle datasets with millions of hypotheses, a critical requirement for modern large-scale inference.

Key Results

  • Princessp, particularly with bounded monotone weights, demonstrates strong empirical performance on GWAS datasets, often increasing the number of significant discoveries compared to unweighted Bonferroni and other weighted methods.
  • In simulations, the subsampling approach for optimization shows high accuracy, agreeing with full-method results within at least two significant digits.
  • The method is computationally efficient, with running times below one second even for datasets with millions of hypotheses.
  • The paper shows that Spjøtvoll weights are monotone for small significance levels, providing a theoretical link between unconstrained optimal weights and the constrained monotone weights implemented in Princessp.

Limitations and Future Work

  • Oracle Weights: The current formulation assumes "oracle" weights, meaning they are derived from prior information that is assumed to be correct. In practice, these prior weights are often estimated from data, and the paper acknowledges that the formal optimality properties of these plug-in weights are beyond the scope of the current work.
  • Prior Information Reliability: While bounded weights help mitigate issues with unreliable prior information, the paper doesn't delve deeply into methods for assessing the quality or reliability of prior information itself.
  • Future Work: The authors suggest that future work could explore more general optimization frameworks and further investigate the impact of different types of prior information and constraints on statistical power and error control. The availability of the open-source MATLAB implementation is intended to facilitate further research and application.

Summary updated

Abstract

Researchers in data-rich disciplines---think of computational genomics and\nobservational cosmology---often wish to mine large bodies of $p$-values looking\nfor significant effects, while controlling the false discovery rate or\nfamily-wise error rate. Increasingly, researchers also wish to prioritize\ncertain hypotheses, for example those thought to have larger effect sizes, by\nupweighting, and to impose constraints on the underlying mining, such as\nmonotonicity along a certain sequence.\n We introduce Princessp, a principled method for performing weighted multiple\ntesting by constrained convex optimization. Our method elegantly allows one to\nprioritize certain hypotheses through upweighting and to discount others\nthrough downweighting, while constraining the underlying weights involved in\nthe mining process. When the $p$-values derive from monotone likelihood ratio\nfamilies like the Gaussian means model, the new method allows exact solution of\nan important optimal weighting problem previously thought to be nonconvex and\ncomputationally infeasible. Our method scales to massive dataset sizes.\n We illustrate the applications of Princessp on a series of standard genomics\ndatasets and offer comparisons with several previous `standard' methods.\nPrincessp offers both ease of operation and the ability to scale to extremely\nlarge problem sizes. The method is available as open-source software from\nhttp://github.com/dobriban/pvalue_weighting_matlab .\n
BibTeX
@inproceedings{dobriban2018weighted,
  author = {Edgar Dobriban},
  title = {{Weighted mining of massive collections of $p$-values by convex\textbackslash\{\}n optimization}},
  booktitle = {PubMed},
  year = {2018},
  doi = {10.1093/imaiai/iax013},
}

Loading related content...

Figure