Global and local CLTs for linear spectral statistics of general sample covariance matrices when the dimension is much larger than the sample size with applications

Xiucai Ding, Zhenggang WangIEEE Trans. Information Theory 2023

TL;DR

This paper introduces new statistical tools (Central Limit Theorems) for analyzing covariance matrices in high-dimensional settings where the number of features ($p$) far exceeds the number of samples ($n$). These tools allow for more powerful hypothesis testing about the structure of these matrices, particularly by developing "local" tests that don't require knowledge of the data's fourth moment, a significant advantage over existing methods.

What Problem Does This Paper Address?

In many fields, understanding the relationships between variables is crucial, and this is often done by studying the covariance matrix. However, in modern datasets, the number of variables ($p$) can be vastly larger than the number of observations ($n$) – a scenario known as "ultra-high dimensionality" ($p \gg n$). Traditional statistical methods struggle in this regime. A key problem is testing whether the true covariance structure ($Σ$) matches a hypothesized structure ($Σ_0$). Existing methods for this problem often rely on "global" tests that consider all eigenvalues of the covariance matrix. While powerful, these global tests typically require knowledge of the fourth cumulant (a measure of data's "tailedness") of the underlying data, which is often unknown and difficult to estimate reliably. This paper addresses this by developing both "global" and "local" statistical tests for covariance structures in the ultra-high dimensional setting, with a particular focus on creating "local" tests that are free from the need to know the fourth cumulant.

Methods

The core of this paper lies in establishing Central Limit Theorems (CLTs) for linear spectral statistics (LSS) of sample covariance matrices. The authors develop two types of CLTs:

  • Global CLTs: These apply to macroscopic scales, considering all eigenvalues of the sample covariance matrix. They establish that these statistics converge to Gaussian processes, but their mean and covariance functions depend on the fourth cumulant of the data.
  • Local CLTs: These apply to mesoscopic scales, focusing on a smaller range of eigenvalues. Crucially, these local CLTs show that the resulting Gaussian processes do not depend on the fourth cumulant. This is a key theoretical breakthrough.

The proofs leverage advanced techniques from random matrix theory, including the Helffer-Sjöstrand formula, cumulant expansions, and Wick's theorem, to handle the complex analytical challenges of the ultra-high dimensional setting.

Datasets

Not discussed in the paper. The paper focuses on theoretical results and uses simulated data for numerical experiments.

Key Insights and Ideas

  • Local vs. Global CLTs: The distinction between global and local CLTs is critical. While global tests capture overall spectral behavior, local tests can zoom in on specific regions of the eigenvalue spectrum.
  • Moment-Free Testing: The most significant insight is the development of local statistical tests that do not require knowledge of the fourth cumulant of the data. This is a major practical advantage, as estimating higher-order moments in high dimensions is notoriously difficult and prone to error.
  • Ultra-High Dimensional Regime: The paper successfully extends existing random matrix theory results to the $p \gg n$ regime ($p \asymp n^\alpha$ with $\alpha > 1$), which is more relevant to many modern datasets than the previously studied $p \approx n$ or $p < n$ settings.
  • New Hypothesis Testing Framework: The paper proposes a novel framework for hypothesis testing about covariance structures, leveraging the newly developed local CLTs.

Key Results

  • Establishment of Global and Local CLTs: The paper rigorously proves both global and local CLTs for linear spectral statistics of sample covariance matrices in the ultra-high dimensional setting ($p \gg n$).
  • Gaussian Convergence: The LSS are shown to converge to Gaussian processes, with their mean and covariance functions explicitly identified.
  • Moment-Free Local Tests: A key result is that the local CLTs and the derived statistics are independent of the fourth cumulant of the data, unlike existing global methods.
  • Powerful Hypothesis Testing: The proposed local statistics demonstrate superior power under general local alternatives compared to existing methods, especially when the fourth cumulant is unknown or misspecified.
  • Numerical Validation: Simulations confirm the accuracy and robustness of the proposed statistics, showing better performance than several existing methods.

Limitations and Future Work

  • Diagonal Covariance Matrix: The paper assumes a diagonal population covariance matrix ($Σ$) for simplicity and interpretability. While the authors suggest this assumption can be removed with additional technical effort, it remains a limitation.
  • Smoothness of Test Functions: The proofs involve certain regularity conditions on the test functions used for LSS. While the paper discusses ways to construct suitable functions, this can be a technical consideration.
  • Generalization of Techniques: The authors mention that their techniques could be extended to study other types of random matrices and potentially weaken regularity conditions on test functions.
  • Further Applications: The paper focuses on hypothesis testing for covariance structures. Future work could explore applications in other areas of multivariate analysis and high-dimensional statistics.

Summary updated

Abstract

In this paper, under the assumption that the dimension is much larger than the sample size, i.e., $p \asymp n^α, α&gt;1,$ we consider the (unnormalized) sample covariance matrices $Q = Σ^{1/2} XX^*Σ^{1/2}$, where $X=(x_{ij})$ is a $p \times n$ random matrix with centered i.i.d entries whose variances are $(pn)^{-1/2}$, and $Σ$ is the deterministic population covariance matrix. We establish two classes of central limit theorems (CLTs) for the linear spectral statistics (LSS) for $Q,$ the global CLTs on the macroscopic scales and the local CLTs on the mesoscopic scales. We prove that the LSS converge to some Gaussian processes whose mean and covariance functions depending on $Σ$, the ratio $p/n$ and the test functions, can be identified explicitly on both macroscopic and mesoscopic scales. We also show that even though the global CLTs depend on the fourth cumulant of $x_{ij},$ the local CLTs do not. Based on these results, we propose two classes of statistics for testing the structures of $Σ,$ the global statistics and the local statistics, and analyze their superior power under general local alternatives. To our best knowledge, the local LSS testing statistics which do not rely on the fourth moment of $x_{ij},$ is used for the first time in hypothesis testing while the literature mostly uses the global statistics and requires the prior knowledge of the fourth cumulant. Numerical simulations also confirm the accuracy and powerfulness of our proposed statistics and illustrate better performance compared to the existing methods in the literature.
BibTeX
@article{ding2023global,
  author = {Xiucai Ding and Zhenggang Wang},
  title = {{Global and local CLTs for linear spectral statistics of general sample covariance matrices when the dimension is much larger than the sample size with applications}},
  journal = {IEEE Transactions on Information Theory},
  year = {2023},
}
Figure