Invariance Principle and Clt for the Spiked Eigenvalues of Large-dimensional Fisher Matrices and Applications

Dandan Jiang, Zhiqiang Hou, Zhidong Bai, Runze Li

This paper, published in the Annals of Applied Probability, provides a significant theoretical leap in Random Matrix Theory (RMT) by establishing an invariance principle and a Central Limit Theorem (CLT) for the spiked eigenvalues of large-dimensional Fisher matrices. By relaxing traditional requirements for Gaussian data and bounded fourth moments, the authors provide a more robust framework for high-dimensional statistical inference, particularly in two-sample covariance testing and signal detection.

TL;DR

The authors prove that the behavior of "spiked" (outlier) eigenvalues in large Fisher matrices is largely independent of the underlying data distribution. They use this "invariance principle" to derive a Central Limit Theorem that works even for heavy-tailed data, significantly improving the accuracy of statistical tests like Roy’s Maximum Root test in high-dimensional settings.

What Problem Does This Paper Address?

In multivariate statistics, comparing two populations often involves analyzing a Fisher matrix ($F = S_1 S_2^{-1}$), where $S_1$ and $S_2$ are sample covariance matrices. When the two populations are mostly similar but differ in a few specific directions (a "low-rank perturbation"), these differences appear as "spiked" eigenvalues—outliers that sit outside the main bulk of the eigenvalue distribution.

Historically, analyzing these spikes required two restrictive assumptions:

  1. Gaussianity: The data must follow a normal distribution.
  2. Bounded Fourth Moments: The data cannot have "heavy tails" (extreme outliers).

However, real-world data (in finance, genomics, or wireless signals) is rarely Gaussian and often heavy-tailed. Furthermore, previous models often assumed the population covariance matrices were diagonal or had very simple structures. This paper addresses the need for a mathematical framework that handles non-Gaussian, heavy-tailed, and complexly structured high-dimensional data.

Methods

The authors establish their results through several key mathematical innovations:

  • The Invariance Principle: They prove that the limiting distribution of spiked eigenvalues remains the same across a wide class of population distributions, provided they satisfy certain mean, variance, and tail probability conditions.
  • Generalized Spiked Model: Unlike previous work that assumed a single "threshold" for spikes, this paper allows for multiple "bulks" of spikes and population covariances that are not diagonal.
  • Relaxed Moment Conditions: Instead of requiring bounded fourth moments (which excludes many heavy-tailed distributions), they use a tail probability condition (Assumption 3). This allows the theory to apply to distributions like the $t$-distribution with 4 degrees of freedom.
  • Truncation and Centralization: To handle the math of non-Gaussian entries, the authors employ sophisticated RMT techniques to "truncate" extreme values and re-normalize the variables without losing the asymptotic properties of the spikes.

Datasets

As a theoretical statistics paper, the authors do not use a specific real-world dataset. Instead, they evaluate their findings using extensive Monte Carlo simulations across three scenarios:

  1. Gaussian Distribution: Standard normal variables.
  2. Binary Distribution: Variables taking values $\pm 1$ with equal probability.
  3. Heavy-Tailed Distribution: Data generated from a $t(4)$ distribution, where the fourth moment is infinite.

Key Insights and Ideas

  • Universality of Spikes: The "aha moment" is that the signal (the spikes) is much more robust than the noise (the bulk). Even if your data is messy and non-normal, the outliers caused by population differences follow predictable, universal laws as the dimension $p$ grows.
  • Roy’s Test Power: The paper provides a way to calculate the power function (the probability of correctly detecting a difference) for Roy’s Maximum Root test. This was previously a "black box" in high dimensions.
  • Phase Transitions: The authors detail the "phase transition" where a spike only becomes detectable once the population difference exceeds a certain critical threshold. They show how this threshold is affected by the ratio of sample sizes ($n_1, n_2$) and the dimension ($p$).

Key Results

  • Superior Accuracy: In simulations, the authors' new CLT-based method consistently outperformed the previous state-of-the-art (Wang and Yao, 2017), especially in non-Gaussian cases.
  • Robustness to Heavy Tails: Their method remained accurate for $t(4)$ distributions, whereas older methods that assumed bounded fourth moments failed to provide reliable results.
  • High-Dimensional Performance: For Roy’s test at $p=200$, the proposed method maintained the correct "size" (false positive rate) while achieving high "power" (detection rate), even when traditional Likelihood Ratio Tests (LRT) became computationally unstable or failed.

Limitations and Future Work

  • Partial Invariance: The authors note that in extremely complex cases where variables have high dependency, a "partial invariance principle" might be required. They opted to omit the full details of this due to space constraints, suggesting it as a path for future refinement.
  • Finite Sample Performance: While the asymptotic theory (as $p \to \infty$) is solid, the authors used a small correction ($p-M$ instead of $p$) in their simulations to improve accuracy for finite samples. Further work could formalize these finite-sample corrections.
  • Computational Complexity: Estimating the population spikes ($\alpha_k$) from sample data requires solving complex equations involving Stieltjes transforms, which may be computationally intensive for extremely large datasets.

Summary updated

Abstract

This paper aims to derive the asymptotic distributions of the spiked eigenvalues of large-dimensional spiked Fisher matrices, without imposing Gaussian assumptions or restrictive assumptions on covariance matrices.We first establish an invariance principle for the spiked eigenvalues of the Fisher matrix.That is, we show that the limiting distributions of the spiked eigenvalues are invariant over a broad range of population distributions satisfying certain conditions.Utilizing this invariance principle, we establish a central limit theorem (CLT) for the spiked eigenvalues, and further explore some interesting applications by using the CLT to derive the power functions of the Roy Maximum root test for linear hypotheses in linear models, as well as the test in signal detection.To evaluate the effectiveness of the newly proposed test, we conduct Monte Carlo simulation studies and compare its performance with existing tests.
BibTeX
@inproceedings{jiang2023invariance,
  author = {Dandan Jiang and Zhiqiang Hou and Zhidong Bai and Runze Li},
  title = {{Invariance Principle and Clt for the Spiked Eigenvalues of Large-dimensional Fisher Matrices and Applications}},
  booktitle = {Statistica},
  year = {2023},
  doi = {10.5705/ss.202022.0377},
}

Loading related content...

Figure