Abstract
We derive a Gaussian approximation result for the maximum of a sum of high-dimensional random vectors. Specifically, we establish conditions under which the distribution of the maximum is approximated by that of the maximum of a sum of the Gaussian random vectors with the same covariance matrices as the original vectors. This result applies when the dimension of random vectors ($p$) is large compared to the sample size ($n$); in fact, $p$ can be much larger than $n$, without restricting correlations of the coordinates of these vectors. We also show that the distribution of the maximum of a sum of the random vectors with unknown covariance matrices can be consistently estimated by the distribution of the maximum of a sum of the conditional Gaussian random vectors obtained by multiplying the original vectors with i.i.d. Gaussian multipliers. This is the Gaussian multiplier (or wild) bootstrap procedure. These distributional approximations yield high-quality approximations to the distribution of the original maximum, often with approximation error decreasing polynomially in the sample size.
Introduction
In classical statistical theory, the Central Limit Theorem (CLT) provides the foundation for asymptotic inference, asserting that the sum of independent random variables converges to a Gaussian distribution as the sample size $n$ increases. However, modern data science frequently encounters "high-dimensional" regimes where the number of variables $p$ is comparable to or significantly exceeds $n$. In such settings, the standard CLT often fails to provide accurate approximations. This paper addresses a critical gap by establishing a Gaussian approximation for the maximum coordinate of a sum of high-dimensional random vectors, even when $p$ grows exponentially with $n$.
Theoretical Framework
Gaussian Approximation Result (GAR)
The authors prove that the distribution of the maximum of a sum of $n$ independent $p$-dimensional random vectors can be closely approximated by the distribution of the maximum of a Gaussian vector with the same mean and covariance. A key innovation is that this approximation holds without requiring specific correlation structures among the $p$ variables. The dimension $p$ can be as large as $e^{n^c}$ for some constant $c > 0$, effectively breaking the traditional $p \ll n$ barrier.
The Multiplier Bootstrap
In practice, the true covariance matrix of the data is rarely known. To address this, the paper validates the "multiplier bootstrap" (or wild bootstrap). This procedure involves multiplying the centered data vectors by independent standard normal multipliers to simulate a conditional Gaussian distribution. The authors demonstrate that this simulated distribution consistently estimates the true distribution of the maximum, providing a computationally feasible path for high-dimensional inference.
The proof of these results relies on advanced probabilistic tools, including Slepian’s smart path interpolation, Stein’s method, and the use of a "smooth max" function to handle the non-differentiability of the maximum operator.
Simulation Results and Empirical Validation
To evaluate the theoretical bounds, the authors conducted Monte Carlo simulations. One key visualization used is the Probability-Probability (P-P) plot, which compares the cumulative distribution function (CDF) of the bootstrap-estimated maximum against the true distribution.
As shown in the figure, the bootstrap approximation is remarkably accurate even for $n=100$ and $p=5000$. The alignment with the 45-degree line, particularly in the upper tails, confirms that the method is well-suited for setting critical values in hypothesis testing.
Applications in High-Dimensional Statistics
Sparse Regression and the Dantzig Selector
The paper applies the GAR to improve the performance of the Dantzig selector in high-dimensional sparse regression. By using a "design-adaptive" penalty level derived from the multiplier bootstrap, the authors achieve tighter error bounds and better variable selection compared to traditional universal penalties.
Multiple Hypothesis Testing
The results provide a robust framework for controlling the family-wise error rate (FWER) in multiple testing scenarios. The method allows for strong control even when the number of hypotheses is exponentially larger than the sample size, without requiring independence between the tests.
Conclusion
This work provides a significant theoretical advancement in high-dimensional statistics. By showing that the maximum of a sum converges to a Gaussian maximum much faster than the entire vector converges to a multivariate Gaussian, the authors enable valid inference in regimes previously thought inaccessible. Future research directions include extending these results to dependent observations (e.g., time series) and relaxing moment requirements for extremely heavy-tailed data.
Summary updated