TL;DR
This paper provides the first comprehensive statistical limit theory for "smoothed" $p$-Wasserstein distances, proving that Gaussian smoothing allows for fast, dimension-independent convergence rates. By leveraging dual Sobolev spaces and the functional delta method, the authors enable valid statistical inference—such as confidence intervals and hypothesis testing—for optimal transport problems in high-dimensional settings.
What Problem Does This Paper Address?
The Wasserstein distance ($W_p$) is a powerful tool in machine learning and statistics for comparing probability distributions. However, it suffers from a notorious "curse of dimensionality": the number of samples needed to estimate the distance accurately grows exponentially with the dimension $d$. For any $d > 2$, the convergence rate is roughly $n^{-1/d}$, which is agonizingly slow for real-world data.
While researchers recently discovered that convolving distributions with a small amount of Gaussian noise (smoothing) can improve this rate to a "parametric" $n^{-1/2}$ (independent of dimension), we lacked a limit distribution theory. Without knowing the shape of the error distribution as the sample size goes to infinity, we couldn't perform basic statistical tasks like building confidence intervals or conducting rigorous hypothesis tests for generative models.
Methods
The authors bridge this gap by treating the smooth Wasserstein distance as a functional on a specific mathematical structure: dual Sobolev spaces.
Key technical steps include:
- Embedding: They embed the domain of the Wasserstein distance into a dual Sobolev space ($H^{-1,p}$), which provides a norm that plays nicely with Gaussian smoothing.
- Hadamard Differentiability: They prove that the smooth $W_p$ distance is "Hadamard directionally differentiable." This is a fancy way of saying the distance changes predictably when you slightly nudge the underlying distributions.
- Extended Functional Delta Method: By combining the differentiability result with the weak convergence of the empirical process in the Sobolev space, they derive the explicit form of the limit distributions.
- Bootstrap Consistency: They prove that the nonparametric bootstrap (resampling your data to estimate error) is mathematically valid for these smoothed distances, even in cases where the derivative is non-linear.
Datasets
As a theoretical contribution to mathematical statistics, the paper does not use empirical datasets or benchmarks (like MNIST or ImageNet). Instead, it provides rigorous mathematical proofs that apply to any Borel probability measures satisfying certain moment conditions (specifically, sub-Gaussian or sub-Weibull distributions).
Key Insights and Ideas
- Smoothing as a Regularizer: Gaussian smoothing doesn't just make the math easier; it fundamentally changes the statistical nature of the problem. It "shrinks" the class of functions we care about, making the empirical process well-behaved in any dimension.
- The $p > 1$ Breakthrough: Previous limit results existed for $W_1$, which can be written as a simple maximum over Lipschitz functions. For $p > 1$, this structure disappears. The authors' use of dual Sobolev spaces is the "key insight" that unlocks the $p > 1$ case.
- Null vs. Alternative: The paper reveals a fascinating split in behavior. When comparing a distribution to itself (the "null" case), the limit distribution is non-Gaussian (the supremum of a Gaussian process). When comparing two different distributions (the "alternative"), the limit is a standard Normal distribution.
- Generative Modeling (MDE): The theory directly applies to Minimum Distance Estimation. The authors show that if you train a model to minimize the smooth $W_2$ distance, the resulting parameters will be asymptotically normal.
Key Results
- Theorem 1.1 (The Main Event): Provides explicit limit laws for one-sample and two-sample cases.
- Under the Null ($\mu = \nu$): $\sqrt{n} W_p^{(\sigma)}(\hat{\mu}_n, \mu)$ converges to the norm of a Gaussian process in the dual Sobolev space.
- Under the Alternative ($\mu \neq \nu$): The error is asymptotically normal: $\sqrt{n}(W_p^{(\sigma)}(\hat{\mu}_n, \nu) - W_p^{(\sigma)}(\mu, \nu)) \xrightarrow{d} N(0, \text{Var})$.
- Bootstrap Validity: The authors establish that the bootstrap consistently estimates these limits, providing a practical way for practitioners to compute p-values or confidence intervals.
- Parametric Rates: Confirms that the smooth distance avoids the curse of dimensionality, maintaining an $O(n^{-1/2})$ convergence rate regardless of how many dimensions the data has.
Limitations and Future Work
- Computational Cost: While the statistical efficiency is high, the computational cost of calculating $W_p$ in high dimensions remains significant. The authors suggest that finding more efficient ways to compute these smoothed distances is a vital next step.
- Choice of $\sigma$: The theory assumes a fixed smoothing parameter $\sigma$. Future work could explore "adaptive" smoothing, where $\sigma$ decreases as the sample size $n$ increases.
- Berry-Esseen Bounds: While we now know the limit distribution, we don't yet know exactly how fast the empirical distance approaches that limit (rate of convergence of the distribution itself).
Summary updated