CLT for random quadratic forms based on sample means and sample covariance matrices

Wenzhi Yang, Yiming Liu, Guangming Pan, Zhou WangarXiv 2022 · DOI · arXiv

TL;DR

This paper establishes a new Central Limit Theorem (CLT) for quadratic forms in high-dimensional settings where the number of variables ($q$) far exceeds the sample size ($n$). By projecting data into a lower-dimensional subspace ($p$), the authors provide the mathematical foundation to perform reliable statistical testing even when traditional methods like Hotelling’s $T^2$ fail due to singular covariance matrices.

What Problem Does This Paper Address?

In classical multivariate statistics, we rely heavily on the sample mean and sample covariance matrix to test hypotheses. However, modern data often suffers from the "curse of dimensionality," where the number of features ($q$) is much larger than the number of observations ($n$).

When $q > n-1$, the sample covariance matrix becomes singular (it has no inverse). This "breaks" fundamental tools like Hotelling’s $T^2$ test, which requires an invertible covariance matrix to measure the distance between a sample mean and a population mean. While researchers have proposed various workarounds, many struggle to provide a rigorous Central Limit Theorem—the "gold standard" for understanding how a statistic behaves as the sample size grows—especially for complex quadratic forms involving both means and covariances.

Methods

The authors employ a dimensionality reduction technique. Instead of working with the unwieldy $q$-dimensional data directly, they project the sample vectors into a $p$-dimensional subspace using a non-random matrix $U_{p \times q}$.

Key aspects of their approach include:

  • Subspace Projection: They assume $p \ll q$, and specifically that the ratio of the projected dimension to the sample size ($p/n$) vanishes as they both go to infinity ($p/n \to 0$).
  • Stochastic Analysis: The proof relies on sophisticated tools from Random Matrix Theory, specifically Martingale Difference Sequences and Stieltjes transforms.
  • Functional Generalization: Rather than proving the CLT for a single specific statistic, they prove it for a general class of analytic functions applied to the sample covariance matrix and quadratic forms of the sample mean.

Datasets

Not discussed in the paper. This is a theoretical probability and mathematical statistics paper. The results are derived through rigorous mathematical proofs rather than empirical simulations or real-world data applications.

Key Insights and Ideas

  • Decoupling Dimensions: The most powerful insight is that the original dimension $q$ can be arbitrarily large—even exponentially larger than the sample size—as long as the data can be projected into a "manageable" $p$-dimensional space where $p$ grows slower than $n$.
  • Joint Convergence: The paper doesn't just look at the mean or the covariance in isolation. It establishes the joint asymptotic normality of two statistics ($X_n$ and $Y_n$) that represent the interaction between the sample mean and the spectral properties of the covariance matrix.
  • The "Aha" Moment: The authors show that even in high dimensions, the "noise" from the extra dimensions can be controlled through projection, allowing the resulting quadratic forms to converge to a predictable bivariate normal distribution.

Key Results

The centerpiece of the paper is Theorem 1, which states that as $\min(p, n) \to \infty$:

  1. The joint distribution of the defined quadratic forms $(X_n, Y_n)$ converges in distribution to a bivariate normal distribution $N(\mathbf{0}, \Gamma_1)$.
  2. The authors provide an explicit form for the covariance matrix $\Gamma_1$, which depends on the derivatives of the functions used in the statistics (specifically $f(1)$ and $g'(0)$).
  3. This result effectively "rescues" the possibility of performing Hotelling-style tests in high-dimensional spaces by providing the necessary distribution theory to calculate p-values and critical values.

Limitations and Future Work

  • The $p/n \to 0$ Constraint: The theory requires the projected dimension $p$ to be significantly smaller than the sample size $n$. It does not cover the "high-dimensional" regime where $p$ and $n$ are of the same order (often called the Marčenko-Pastur regime).
  • Non-random Projection: The matrix $U$ is assumed to be non-random and fixed. Exploring the effects of data-dependent (random) projections, like those used in compressed sensing, remains an open area.
  • Application to Testing: While this paper provides the theory, the authors note that a follow-up paper will specifically apply these results to derive a practical, high-dimensional version of the Hotelling’s $T^2$ test.

Summary updated

Abstract

In this paper, we use the dimensional reduction technique to study the central limit theory (CLT) random quadratic forms based on sample means and sample covariance matrices. Specifically, we use a matrix denoted by $U_{p\times q}$, to map $q$-dimensional sample vectors to a $p$ dimensional subspace, where $q\geq p$ or $q\gg p$. Under the condition of $p/n\rightarrow 0$ as $(p,n)\rightarrow \infty$, we obtain the CLT of random quadratic forms for the sample means and sample covariance matrices.
BibTeX
@misc{yang2022clt,
  author = {Wenzhi Yang and Yiming Liu and Guangming Pan and Zhou Wang},
  title = {{CLT for random quadratic forms based on sample means and sample covariance matrices}},
  year = {2022},
  eprint = {2210.11215},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.11215},
}

Loading related content...

Figure