A Unified Theory of Confidence Regions and Testing for High-Dimensional Estimating Equations

Matey Neykov, Yang Ning, Jun S. Liu, Han Liu

TL;DR

This paper introduces a novel, likelihood-free framework for constructing confidence regions and performing hypothesis tests in high-dimensional statistical models defined by estimating equations. The key innovation is a projection-based method that allows for valid inference in a broad range of problems where traditional likelihood-based approaches are difficult or impossible to apply.

What Problem Does This Paper Address?

In high-dimensional statistics, where the number of parameters ($d$) can exceed the number of observations ($n$), many standard statistical inference methods break down. Estimating equations, which define relationships between data and parameters through moment conditions, are a powerful tool for modeling. However, constructing reliable confidence intervals and performing hypothesis tests for parameters in these models becomes challenging when $d \gg n$. Existing methods often rely on likelihood functions, which are not always available or easy to specify. This paper addresses the need for a unified, likelihood-free inferential framework that can handle high-dimensional estimating equations across a diverse set of statistical problems.

Methods

The core of the proposed method involves projecting the high-dimensional estimating equation onto a sparse direction. This projection is achieved by solving a large-scale linear program, which yields a sparse direction that effectively captures the influence function of the estimating equation. This projected estimating equation is then used to construct a Z-estimator for the parameter of interest. The theoretical contribution lies in establishing a unified Z-estimation theory for confidence regions in high-dimensional settings. This framework is designed to be "likelihood-free," meaning it does not require the specification of a likelihood or pseudo-likelihood function, making it applicable to a wider range of problems. The authors provide detailed theoretical results and demonstrate the applicability of their framework to various examples, including noisy compressed sensing, instrumental variable regression, graphical models, discriminant analysis, and vector autoregressive models.

Datasets

The paper does not introduce new datasets. Instead, it uses a real dataset for gene network analysis from the mouse aging project (Zahn et al., 2007) to demonstrate the practical application of their proposed method.

Key Insights and Ideas

  • Likelihood-Free Inference: The most significant insight is the development of a unified inferential framework that bypasses the need for likelihood functions, opening up inference for models where likelihoods are intractable or non-existent.
  • Projection onto Sparse Directions: The novel approach of projecting estimating equations onto sparse directions is a key technical innovation that allows for the construction of valid confidence regions in high dimensions.
  • Unified Theory: The paper establishes a general Z-estimation theory that unifies inference across a broad class of high-dimensional estimating equation problems, demonstrating its versatility.
  • Broad Applicability: The framework is shown to be effective for a wide range of challenging problems, including those with instrumental variables, graphical models, and autoregressive processes, where existing methods may struggle.

Key Results

The paper presents theoretical results establishing the consistency and asymptotic normality of their proposed Z-estimator. They demonstrate that their method provides valid confidence intervals and hypothesis tests for parameters in high-dimensional estimating equation models. Numerical simulations and a real-data analysis are conducted to support the theoretical findings. The results show that their method achieves coverage probabilities close to the nominal level and often results in shorter confidence intervals compared to existing methods, particularly in high-dimensional settings.

Limitations and Future Work

The authors acknowledge that their method, while general, focuses on constructing confidence intervals for a single parameter of interest. Extending this to obtain global honest confidence regions for the entire parameter vector $\beta$ is identified as an interesting problem for future investigation. They also mention that their results can be extended to cases with multi-dimensional parameters of interest.

Summary updated

Abstract

We propose a new inferential framework for constructing confidence regions and testing hypotheses in statistical models specified by a system of high-dimensional estimating equations. We construct an influence function by projecting the fitted estimating equations to a sparse direction obtained by solving a large-scale linear program. Our main theoretical contribution is to establish a unified Z-estimation theory of confidence regions for high-dimensional problems. Different from existing methods, all of which require the specification of the likelihood or pseudo-likelihood, our framework is likelihood-free. As a result, our approach provides valid inference for a broad class of high-dimensional constrained estimating equation problems, which are not covered by existing methods. Such examples include, noisy compressed sensing, instrumental variable regression, undirected graphical models, discriminant analysis and vector autoregressive models. We present detailed theoretical results for all these examples. Finally, we conduct thorough numerical simulations, and a real dataset analysis to back up the developed theoretical results.
BibTeX
@inproceedings{neykov2018unified,
  author = {Matey Neykov and Yang Ning and Jun S. Liu and Han Liu},
  title = {{A Unified Theory of Confidence Regions and Testing for High-Dimensional Estimating Equations}},
  booktitle = {Statistical},
  year = {2018},
  doi = {10.1214/18-sts661},
}
Figure