In classical statistics, the Central Limit Theorem (CLT) is the ultimate safety net. It guarantees that the average of $n$ independent random variables will eventually converge to a normal (Gaussian) distribution as the sample size $n$ approaches infinity. This property underpins almost all traditional hypothesis testing, confidence intervals, and error bounds.
However, the classical CLT assumes that the number of features or dimensions ($p$) remains fixed as the sample size ($n$) grows. In modern data regimes—such as genomics, finance, and high-dimensional machine learning—we routinely encounter the "large $p$, small $n$" paradigm ($p \gg n$). When the dimensionality grows exponentially faster than the sample size, the classical geometric and probabilistic guarantees collapse. The error in the standard normal approximation accumulates across dimensions, and estimating the massive $p \times p$ covariance matrix becomes mathematically impossible because we lack the degrees of freedom to make it non-singular.
To perform reliable inference on modern datasets, researchers have developed a powerful framework: High-Dimensional Central Limit Theorems and Gaussian Approximations. This paradigm proves that under proper scaling and smoothing, valid asymptotic normality and distribution-free inference can be recovered even when $p$ is exponentially larger than $n$.
The Core Mechanism: Approximating the Maximum
If forcing an entire $p$-dimensional joint distribution to converge to a multivariate normal distribution requires $n \gg p$, how can we operate when $p \gg n$?
The key insight is to shift our focus from the entire joint distribution to specific, highly informative functionals of the data. The most vital of these is the maximum coordinate of a sum of random vectors:
$$\max_{1 \le j \le p} \frac{1}{\sqrt{n}} \sum_{i=1}^n X_{ij}$$
This statistic is central to multiple hypothesis testing, simultaneous confidence bands, and anomaly detection. In a series of foundational papers, Denis Chetverikov et al. demonstrated that the distribution of this maximum coordinate can be closely coupled with the maximum of a true Gaussian vector, even if $\log p$ scales as a power of $n$ (i.e., $p$ grows exponentially relative to $n$).
In Central limit theorems and bootstrap in high dimensions and Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors, the authors proved that the Kolmogorov distance between the distribution of the maximum of the random variables and the maximum of the corresponding Gaussian vector decays to zero asymptotically.
Why the Maximum Behaves Nicely: Anti-Concentration
For a Gaussian approximation of a maximum to be useful, the probability density of the Gaussian vector's maximum cannot be arbitrarily concentrated in a tiny region. If the distribution had sharp spikes, a tiny error in the approximation would lead to a massive error in the calculated p-values or confidence levels.
To solve this, Kengo Kato et al. derived novel anti-concentration bounds in Comparison and anti-concentration bounds for maxima of Gaussian random vectors. They proved that the maximum of a Gaussian vector cannot concentrate too heavily in any narrow interval, regardless of how complex or highly correlated the underlying variables are. This geometric stability provides the mathematical foundation that allows high-dimensional CLTs to remain robust under arbitrary correlation structures.
Bypassing Covariance Estimation via the Multiplier Bootstrap
Even if we know that the maximum of our high-dimensional sum behaves like the maximum of a Gaussian vector, we still face a practical roadblock: the target Gaussian distribution depends entirely on the true $p \times p$ covariance matrix $\Sigma$ of our data. When $p \gg n$, the empirical covariance matrix is singular and a poor estimator of $\Sigma$.
The breakthrough that makes this theory actionable is the multiplier bootstrap (sometimes called the wild bootstrap). Instead of explicitly estimating $\Sigma$, the authors in Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors showed that we can simulate the target distribution directly from our data samples.
The mechanism is elegant:
- Take the observed data vectors $X_1, X_2, \dots, X_n$.
- Generate independent, standard normal scalar weights $\xi_1, \xi_2, \dots, \xi_n \sim \mathcal{N}(0, 1)$.
- Construct the bootstrapped sum: $\frac{1}{\sqrt{n}} \sum_{i=1}^n \xi_i X_i$.
Because the weights $\xi_i$ are independent and have a variance of 1, the conditional covariance of this bootstrapped sum given the data is exactly equal to the sample covariance matrix of the original data. By computing the maximum coordinate of this bootstrapped sum repeatedly, we can accurately map out the critical values and tail probabilities of the target Gaussian distribution. This completely bypasses the need to invert or explicitly regularize a massive covariance matrix.
To push this further, Nazar Buzun et al. utilized optimal transport theory and Wasserstein distances in Strong Gaussian Approximation for the Sum of Random Vectors to derive significantly tighter bounds. Their work dramatically lowers the practical sample size $n$ required to guarantee that these bootstrap approximations remain valid.
Universal Asymptotics and Dimension-Free Convergence
Another facet of this research direction focuses on designing test statistics that converge to a standard normal distribution regardless of the relationship between $p$ and $n$. This is known as a universal asymptotic framework.
Distribution-Free Scaling
Rather than restricting how fast the feature space can grow, researchers use clever centering and scaling transformations. For example, Christophe Ley et al. demonstrated in High-dimensional tests for spherical location and spiked covariance that directional statistics, which often exhibit severe skewness in low dimensions, naturally converge toward a standard normal distribution as the number of dimensions increases. By applying a universal scaling procedure, their test statistics achieve standard normality without requiring any fixed ratio or growth rate constraints between $p$ and $n$.
The Power of U-Statistics
When evaluating complex relationships or performing two-sample testing, classical multivariate alternatives like the $F$-test break down in high dimensions. To circumvent this, researchers rely on U-statistics, which generalize the concept of sample averages to pairs or tuples of data points.
- In A Kernel Method for the Two-Sample-Problem, Arthur Gretton et al. leveraged U-statistics within Reproducing Kernel Hilbert Spaces (RKHS) to create the Maximum Mean Discrepancy (MMD). They showed that asymptotic tests based on U-statistics maintain exceptional sensitivity and predictable normal limits in high-dimensional, low-sample-size environments.
- Similarly, Ping-Shou Zhong demonstrated in Tests for High-Dimensional Regression Coefficients With Factorial Designs that U-statistics provide a robust hypothesis-testing framework for regression coefficients. This approach adapts seamlessly to varying dimensionality without requiring explicit assumptions about the growth rates of predictors versus observations.
Applications Across Complex Statistical Models
The theoretical guarantees of high-dimensional Gaussian approximations act as an engine, enabling valid inference across a wide variety of advanced statistical architectures where traditional methods fail.
High-Dimensional CLT & Bootstrap Engine
│
├─► Robust Estimators (Spatial Median for heavy tails)
├─► Nonparametric Models (Honest Random Forests)
├─► Latent Variable Models (High-Dimensional EM)
└─► Dependent/Functional Data (Time Series & Panels)
Robust Inference and Heavy Tails
When datasets suffer from heavy tails or outliers, the sample mean is highly unstable. In Statistical Inference For Ultrahigh Dimensional Location Parameter Based On Spatial Median, Guanghui Cheng et al. extended Gaussian and bootstrap approximation theories to the spatial median. They proved that robust location parameters maintain asymptotic validity even in ultrahigh-dimensional regimes where $p$ grows exponentially relative to $n$, providing a theoretical green light for constructing confidence intervals around robust estimators.
Nonparametric and Latent Variable Models
- Random Forests: In Estimation and Inference with Trees and Forests in High Dimensions, Vasilis Syrgkanis et al. showed that sub-sampled "honest" regression forests naturally adapt to underlying sparsity. They proved that these algorithms achieve asymptotic normality because their error rates scale primarily with the number of truly relevant features rather than the total dimension $p$.
- The EM Algorithm: In high-dimensional latent variable models, calculating p-values for individual parameters is notoriously difficult. Zhaoran Wang et al. solved this in High Dimensional EM Algorithm: Statistical Optimization and Asymptotic Normality by constructing decorrelated score statistics that isolate individual parameters and exhibit clean asymptotic normality, enabling rigorous hypothesis testing within the EM framework.
Functional and Dependent Data
The framework also extends beyond independent, identically distributed (i.i.d.) data vectors:
- Functional Panels: When dealing with infinite-dimensional data observed over time (such as continuous temperature profiles across multiple stations), Piotr Kokoszka et al. established a high-dimensional CLT in A randomness test for functional panels. Their test statistic maintains asymptotic normality even as the number of series and the number of principal components grow alongside the sample size.
- Moment Restrictions: For time-series data with complex dependencies, Jinyuan Chang et al. adapted traditional over-identification tests in High dimensional generalized empirical likelihood for moment restrictions with dependent data. They showed that by approximating the distribution of the test statistic as the number of moment restrictions grows, model constraints can be validated even when the parameter space is massive relative to the observation window.
By shifting the focus from full multivariate convergence to functional approximations (like the maximum) and leveraging multiplier bootstraps, this research direction provides the mathematical justification needed to trust statistical tests in the high-dimensional settings that define modern data science.
Key Foundational Work
Marginal asymptotics for the “large $p$, small $n$” paradigm: With applications to microarray data (2007) — contributed 1 concept elements to this direction.
A Kernel Method for the Two-Sample-Problem (2006) — contributed 1 concept elements to this direction.