Universality for the largest eigenvalue of a class of sample covariance matrices

Zhigang Bao, Guangming Pan, Zhou WangarXiv 2013 · arXiv

Abstract

This paper establishes the universality of the largest eigenvalue for a broad class of high-dimensional real or complex sample covariance matrices of the form $\mathcal{W}_N=\Sigma^{1/2}XX^*\Sigma^{1/2}$. We consider the regime where the number of observations $N$ and the number of features $M$ both grow to infinity such that $N/M \rightarrow d \in (0, \infty)$. By employing a Green function comparison strategy, we demonstrate that the limiting behavior of the largest eigenvalue is universal for a general class of deterministic positive-definite population covariance matrices $\Sigma$. Specifically, we show that the normalized largest eigenvalue converges to the Tracy-Widom distribution—Type 2 for complex matrices and Type 1 for real matrices with subcritical spikes—extending previous results from the null case ($\Sigma=I$) to more general settings.

Introduction

In multivariate statistics and signal processing, the spectral properties of sample covariance matrices are essential for understanding the structure of high-dimensional data. A central question in random matrix theory is universality: the idea that the local statistics of eigenvalues (such as the largest eigenvalue) should depend only on the symmetry class and the first few moments of the underlying distribution, rather than the specific distribution of the matrix entries.

While universality has been well-established for Wigner matrices and "null" sample covariance matrices (where the population covariance $\Sigma$ is the identity matrix), the case of general $\Sigma$ presents significant technical challenges. This paper bridges that gap, providing a rigorous proof of universality for the largest eigenvalue under a general population covariance structure.

The Matrix Model

The study focuses on the sample covariance matrix $\mathcal{W}_N = \Sigma^{1/2}XX^*\Sigma^{1/2}$, where:

  • $X$ is an $M \times N$ random matrix with independent entries $x_{ij}$.
  • $\mathbb{E}x_{ij} = 0$ and $\mathbb{E}|x_{ij}|^2 = 1/N$.
  • $\Sigma$ is a deterministic $M \times M$ positive-definite matrix.
  • The dimensions $M$ and $N$ are comparable ($M/N \to d$).

Methodology: Green Function Comparison

The authors utilize a sophisticated Green function comparison strategy, originally pioneered by Erdős, Yau, and Yin. The proof relies on two primary pillars:

  1. Strong Local Marčenko-Pastur Law: The authors derive a high-precision estimate of the eigenvalue density near the spectral edge. This involves a "bootstrap" procedure that refines a weak law into a strong law, proving that the eigenvalues are "rigid"—meaning they stay extremely close to their predicted classical locations with very high probability.
  2. Comparison Theorem: By comparing the Stieltjes transform (the trace of the Green function) of the general matrix ensemble to a Gaussian counterpart, the authors show that the distributions of their largest eigenvalues must coincide. A key innovation here is the use of a Lindeberg swapping process at the column level, which preserves the structure imposed by the non-identity $\Sigma$.

Key Insights and Conditions

The degree of universality depends on the complexity of the population covariance matrix $\Sigma$:

  • Condition A (Diagonal $\Sigma$): If $\Sigma$ is diagonal, matching the first two moments (mean and variance) of the entries $x_{ij}$ is sufficient to ensure universality.
  • Condition B (General $\Sigma$): For general deterministic $\Sigma$, the entries must match the first four moments of a Gaussian distribution to guarantee the same limiting behavior.

Furthermore, the paper proves that the self-consistent equations governing the spectral density remain stable even at the very edge of the spectrum, where the density of states vanishes like a square root.

Main Results

The paper provides definitive limits for the largest eigenvalue after appropriate normalization:

  • Complex Case: For complex sample covariance matrices (where $\mathbb{E}x_{ij}^2=0$), the largest eigenvalue converges weakly to the Type 2 Tracy-Widom distribution ($TW_2$).
  • Real Case with Spikes: In the real case, the authors consider "spiked" models where $\Sigma$ has a fixed number of subcritical eigenvalues (outliers that do not yet "pop out" of the bulk). They show that the Type 1 Tracy-Widom distribution ($TW_1$) holds, extending earlier results that were limited to diagonal $\Sigma$.
  • Joint Universality: The results are not limited to the single largest eigenvalue; they extend to the joint distribution of the top $k$ eigenvalues for any fixed $k$.

Discussion and Limitations

While this work significantly advances the understanding of non-null sample covariance matrices, some constraints remain:

  • Tail Conditions: The proof assumes sub-exponential decay for the entries $x_{ij}$, which excludes heavy-tailed data often found in financial contexts.
  • BBP Transition: The study focuses on the bulk edge and subcritical spikes. The "BBP transition"—the phase transition where spikes become large enough to create isolated eigenvalues—under these general conditions remains a subject for future research.
  • Independence: The model assumes independence between entries $x_{ij}$, whereas many real-world applications involve temporal or spatial dependencies that require further investigation.

Summary updated

Abstract

This paper is aimed at deriving the universality of the largest eigenvalue of a class of high-dimensional real or complex sample covariance matrices of the form $\mathcal{W}_N=\Sigma^{1/2}XX^*\Sigma ^{1/2}$. Here, $X=(x_{ij})_{M,N}$ is an $M\times N$ random matrix with independent entries $x_{ij},1\leq i\leq M,1\leq j\leq N$ such that $\mathbb{E}x_{ij}=0$, $\mathbb{E}|x_{ij}|^2=1/N$. On dimensionality, we assume that $M=M(N)$ and $N/M\rightarrow d\in(0,\infty)$ as $N\rightarrow\infty$. For a class of general deterministic positive-definite $M\times M$ matrices $\Sigma$, under some additional assumptions on the distribution of $x_{ij}$'s, we show that the limiting behavior of the largest eigenvalue of $\mathcal{W}_N$ is universal, via pursuing a Green function comparison strategy raised in [Probab. Theory Related Fields 154 (2012) 341-407, Adv. Math. 229 (2012) 1435-1515] by Erdős, Yau and Yin for Wigner matrices and extended by Pillai and Yin [Ann. Appl. Probab. 24 (2014) 935-1001] to sample covariance matrices in the null case ($\Sigma=I$). Consequently, in the standard complex case ($\mathbb{E}x_{ij}^2=0$), combing this universality property and the results known for Gaussian matrices obtained by El Karoui in [Ann. Probab. 35 (2007) 663-714] (nonsingular case) and Onatski in [Ann. Appl. Probab. 18 (2008) 470-490] (singular case), we show that after an appropriate normalization the largest eigenvalue of $\mathcal{W}_N$ converges weakly to the type 2 Tracy-Widom distribution $\mathrm{TW}_2$. Moreover, in the real case, we show that when $\Sigma$ is spiked with a fixed number of subcritical spikes, the type 1 Tracy-Widom limit $\mathrm{TW}_1$ holds for the normalized largest eigenvalue of $\mathcal {W}_N$, which extends a result of Feral and Peche in [J. Math. Phys. 50 (2009) 073302] to the scenario of nondiagonal $\Sigma$ and more generally distributed $X$.
BibTeX
@misc{bao2013universality,
  author = {Zhigang Bao and Guangming Pan and Zhou Wang},
  title = {{Universality for the largest eigenvalue of a class of sample covariance matrices}},
  year = {2013},
  eprint = {1304.5690},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1304.5690},
}
Figure