Singularity-agnostic incomplete U-statistics for testing polynomial constraints in Gaussian covariance matrices

Dennis Y.c. Leung, Nils SturmaarXiv 2024 · DOI · arXiv

Introduction

Testing the goodness-of-fit of models through functional constraints on parameters has a long history, dating back to Spearman’s (1927) analysis of "tetrad" polynomials in factor models. In modern statistics, the Wald test is the standard tool for operationalizing these tests. However, despite its widespread use, the Wald test can produce highly inaccurate test sizes even when classical regularity conditions are met and sample sizes are large. This paper investigates this phenomenon in the context of polynomial constraints in Gaussian covariance matrices and proposes a robust alternative.

The Challenge of Near-Singularity

The primary issue identified is "near-singularity." When the null hypothesis is true but close to a point where the gradient of the constraint function vanishes (a singularity), the standardized Wald test exhibits extremely slow weak convergence. This is caused by the complex dependency structure inherent in the "complete" U-statistics that drive the test's limiting distribution. In finite samples, this leads to p-values that are significantly biased, as the normal approximation fails to hold. This can be rigorously explained by a key ratio of moments encoded in the Berry-Esseen bound quantifying the normal approximation error.

Proposed Methodology: Incomplete U-statistics

To address this, the authors advocate for the use of incomplete U-statistics. Unlike complete U-statistics, which calculate a kernel function for every possible combination of data points, incomplete U-statistics randomly sample a subset of these combinations (e.g., via Bernoulli sampling) based on a computational budget.

By intentionally reducing the number of dependencies, the incomplete U-statistic behaves more like a sum of independent variables, which facilitates faster and more stable convergence to a normal distribution. This approach is "singularity-agnostic," meaning its performance remains reliable regardless of how close the model is to a singular point.

Theoretical Framework

The paper develops a new, refined Berry-Esseen bound to quantify the error in the normal approximation for these statistics. The authors utilize several advanced mathematical techniques:

  • Hoeffding Decomposition: To simplify the statistic into orthogonal components.
  • Hypercontractivity: Leveraging Gaussian polynomial properties to control higher-order moments.
  • Sub-Weibull Bernstein Inequalities: To manage the tail behavior of the statistics.

The resulting bound is $O(n^{-1/2})$ (ignoring logarithmic factors), which is the optimal rate for convergence and a significant improvement over previous $n^{-1/3}$ bounds in the literature.

Empirical Validation

The theoretical findings are supported by simulations using Gaussian data to test tetrad constraints in a 4x4 covariance matrix.

Comparison of empirical test sizes for standard Wald tests versus incomplete U-statistics under near-singular conditions.

As shown in Figure 1, the standard Wald tests ($T_f$ and $\hat{T}_f$) deviate sharply from the ideal diagonal line at a sample size of $n=100$, indicating poor calibration and inaccurate test sizes. In contrast, the standardized incomplete U-statistics (represented by the green markers) align closely with the nominal levels, demonstrating their robustness and "singularity-agnostic" nature in near-singular scenarios.

Conclusion and Future Directions

This research provides a robust solution for testing polynomial constraints in Gaussian models. By moving to incomplete U-statistics, researchers can avoid the pitfalls of the Wald test in near-singular regions.

Future work includes relaxing the Gaussian assumption to broader classes of distributions, establishing formal mathematical bounds for the Studentized case (where variance is estimated), and adapting the framework for high-dimensional multivariate settings with multiple simultaneous constraints.

Summary updated

Abstract

Testing the goodness-of-fit of a model with its defining functional constraints in the parameters could date back to Spearman (1927), who analyzed the famous "tetrad" polynomial in the covariance matrix of the observed variables in a single-factor model. Despite its long history, the Wald test typically employed to operationalize this approach could produce very inaccurate test sizes in many situations, even when the regular conditions for the classical normal asymptotics are met and a very large sample is available. Focusing on testing a polynomial constraint in a Gaussian covariance matrix, we obtained a new understanding of this baffling phenomenon: When the null hypothesis is true but "near-singular", the standardized Wald test exhibits slow weak convergence, owing to the sophisticated dependency structure inherent to the underlying U-statistic that ultimately drives its limiting distribution; this can also be rigorously explained by a key ratio of moments encoded in the Berry-Esseen bound quantifying the normal approximation error involved. As an alternative, we advocate the use of an incomplete U-statistic to mildly tone down the dependence thereof and render the speed of convergence agnostic to the singularity status of the hypothesis. In parallel, we develop a Berry-Esseen bound that is mathematically descriptive of the singularity-agnostic nature of our standardized incomplete U-statistic, using some of the finest exponential-type inequalities in the literature.
BibTeX
@misc{leung2024singularity,
  author = {Dennis Y.C. Leung and Nils Sturma},
  title = {{Singularity-agnostic incomplete U-statistics for testing polynomial constraints in Gaussian covariance matrices}},
  year = {2024},
  eprint = {2401.02112},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2401.02112},
}

Loading related content...

Figure