Authors: Ritwik Sadhu, Ziv Goldfeld, and Kengo Kato
Abstract
The smooth 1-Wasserstein distance (SWD) $W_1^\sigma$ was recently proposed as a means to mitigate the curse of dimensionality in empirical approximation while preserving the Wasserstein structure. Indeed, SWD exhibits parametric convergence rates and inherits the metric and topological structure of the classic Wasserstein distance. Motivated by the above, this work conducts a thorough statistical study of the SWD, including a high-dimensional limit distribution result for empirical $W_1^\sigma$, bootstrap consistency, concentration inequalities, and Berry-Esseen type bounds. The derived nondegenerate limit stands in sharp contrast with the classic empirical $W_1$, for which a similar result is known only in the one-dimensional case. We also explore asymptotics and characterize the limit distribution when the smoothing parameter $\sigma$ is scaled with $n$, converging to $0$ at a sufficiently slow rate. The dimensionality of the sampled distribution enters empirical SWD convergence bounds only through the prefactor (i.e., the constant). We provide a sharp characterization of this prefactor's dependence on the smoothing parameter and the intrinsic dimension. This result is then used to derive new empirical convergence rates for classic $W_1$ in terms of the intrinsic dimension. As applications of the limit distribution theory, we study two-sample testing and minimum distance estimation (MDE) under $W_1^\sigma$. We establish asymptotic validity of SWD testing, while for MDE, we prove measurability, almost sure convergence, and limit distributions for optimal estimators and their corresponding $W_1^\sigma$ error. Our results suggest that the SWD is well suited for high-dimensional statistical learning and inference.
Introduction
Optimal Transport (OT) and the 1-Wasserstein distance ($W_1$) have become indispensable tools in modern machine learning, particularly for tasks involving generative modeling and distribution alignment. However, the practical application of $W_1$ is frequently hindered by the "curse of dimensionality." When estimating $W_1$ from $n$ samples in $d$ dimensions, the error typically scales at a rate of $n^{-1/d}$. In high-dimensional settings, this necessitates an astronomical amount of data to achieve reasonable accuracy.
Furthermore, the classic $W_1$ lacks a general high-dimensional Central Limit Theorem (CLT). Without a known limit distribution, it is difficult to perform rigorous statistical inference, such as hypothesis testing or constructing confidence intervals for model parameters. This paper addresses these gaps by providing a comprehensive statistical foundation for the Smooth 1-Wasserstein Distance (SWD), denoted as $W_1^\sigma$.
The Smooth 1-Wasserstein Distance
The SWD is defined by convolving the target distributions with a Gaussian kernel of scale $\sigma$ before calculating the distance. This smoothing operation acts as a fundamental statistical regularizer. While the classic $W_1$ relies on a search over all 1-Lipschitz functions, the SWD effectively restricts this search to a "smoother" class of functions.
Overcoming the Curse of Dimensionality
The authors prove that by introducing this Gaussian noise, the SWD achieves a parametric convergence rate of $n^{-1/2}$ regardless of the ambient dimension $d$. This is a significant departure from the $n^{-1/d}$ rate of the classic $W_1$. The dimensionality of the space only affects the constant (prefactor) of the convergence, rather than the exponent.
Crucially, the paper demonstrates that the SWD is sensitive to the intrinsic dimension of the data. If a distribution is supported on a low-dimensional manifold embedded within a high-dimensional space, the SWD convergence rate adapts to that lower dimension. This result provides a theoretical justification for why OT-based methods can remain effective even when the ambient dimensionality is high.
Theoretical Breakthroughs
High-Dimensional Limit Distribution
The core technical contribution is the derivation of a high-dimensional CLT for the empirical SWD. By employing Kantorovich-Rubinstein (KR) duality, the authors represent the distance as a supremum over a class of 1-Lipschitz functions. They prove that the class of Gaussian-smoothed Lipschitz functions is P-Donsker.
This property ensures that the empirical process associated with the SWD is stable and converges in distribution to the supremum of a tight Gaussian process. This is the first such result for a Wasserstein-like distance in dimensions $d > 1$, providing a nondegenerate limit that allows for sophisticated uncertainty quantification.
Noise Annealing and Asymptotics
The study also explores the behavior of the SWD when the smoothing parameter $\sigma$ is not fixed but is instead allowed to vanish as the sample size $n$ increases. This "noise annealing" allows the SWD to approach the classic $W_1$ while maintaining the $n^{-1/2}$ convergence rate, provided $\sigma$ converges to zero at a sufficiently slow rate. This bridges the gap between the statistically stable SWD and the geometrically precise classic $W_1$.
Applications in Statistical Inference
Two-Sample Testing
The limit distribution theory enables the development of a consistent framework for two-sample testing. To test the null hypothesis $H_0: P = Q$, the authors propose using the empirical SWD as a test statistic. Because the limit distribution is known, and the nonparametric bootstrap is proven to be consistent for this distance, researchers can now calibrate critical values and perform valid hypothesis testing on high-dimensional datasets.
Minimum Distance Estimation (MDE)
The paper also investigates Minimum Distance Estimators (MDE), which are central to implicit generative models like GANs. The authors establish that estimators which minimize the SWD are measurable and almost surely consistent. Furthermore, they derive the limit distributions for both the optimal estimators and their corresponding SWD error, providing a rigorous path for analyzing the convergence of generative models.
Conclusion
By establishing the limit distribution theory for the Smooth 1-Wasserstein Distance, this work transforms a powerful geometric tool into a robust statistical one. The SWD preserves the desirable properties of Optimal Transport while escaping the curse of dimensionality and enabling valid high-dimensional inference. This framework paves the way for more reliable and theoretically grounded applications of OT in machine learning and statistics.
Reference: Sadhu, R., Goldfeld, Z., & Kato, K. (2022). Limit Distribution Theory for the Smooth 1-Wasserstein Distance with Applications. arXiv:2107.13494v5.
Summary updated