Abstract
We give a comprehensive description of Wasserstein gradient flows of maximum mean discrepancy (MMD) functionals $F_{\nu}:= \text{MMD}^2_K(\cdot,\nu)$ towards given target measures $\nu$ on the real line, focusing on the negative distance kernel $K(x,y):=−|x−y|$. In one dimension, the Wasserstein-2 space can be isometrically embedded into the cone of quantile functions, leading to a characterization of Wasserstein gradient flows via the solution of an associated Cauchy problem on $L_2(0,1)$. We provide a solution to this problem, proving invariance and smoothing properties. For certain flows, initial point measures instantly become absolutely continuous.
Introduction: The Challenge of Non-Smooth Kernels
In generative modeling, transforming a source distribution into a target distribution is often framed as a minimization problem. Wasserstein gradient flows provide a path of steepest descent to minimize cost functionals like the Maximum Mean Discrepancy (MMD). While MMD is widely used with smooth kernels (e.g., Gaussian), this paper investigates the more mathematically challenging negative distance kernel ($K(x,y) = -|x-y|$).
The non-smooth nature of this kernel introduces complexities: flows can exhibit abrupt transitions, such as a point mass "exploding" into a continuous density. Understanding these dynamics is vital for ensuring that generative models produce continuous, high-quality data rather than discrete artifacts.
Methodology: Quantile Embedding and Cauchy Problems
The authors simplify the problem in one dimension by leveraging an isometric embedding. The space of 1D probability measures is mapped to the space of quantile functions in $L_2(0,1)$. This transformation converts the complex Wasserstein distance into a simpler Euclidean-like distance.
By reformulating the gradient flow as a Cauchy problem (a differential inclusion) on these quantile functions, the authors can explicitly calculate the subdifferential of the MMD functional. This allows for rigorous proofs of existence and uniqueness. Furthermore, they introduce an Implicit Euler scheme that decomposes the problem in quantile space, allowing each point's update to be solved independently via a bisection algorithm.
Key Insights: Instant Smoothing and Support Monotonicity
A primary discovery is the instant smoothing effect. For specific MMD flows, a discrete point measure becomes absolutely continuous (smooth) the moment the flow begins. This is caused by the repulsive interaction energy inherent in the MMD functional.
Additionally, the authors prove several key properties:
- Piecewise Linearity: For discrete target measures, the flow is piecewise linear in time at each quantile point.
- Support Monotonicity: If the initial distribution has convex support, it remains convex and grows monotonically to match the target.
- Shape Non-Preservation: Unlike other flows, MMD flows do not preserve distribution families (e.g., a Gaussian-to-Gaussian flow does not remain Gaussian; it flattens before peaking again).
Results and Numerical Stability
The theoretical framework is validated through numerical simulations using synthetic 1D distributions. The implicit Euler scheme demonstrated high stability and accuracy across various cases, including Gaussians, Laplacians, and complex mixtures. The simulations confirm the predicted "smoothing" and "sticking" behaviors, providing a clear visual representation of how mass redistributes over time.
Conclusion and Future Work
This work provides the first rigorous description of distance-kernel MMD gradient flows in 1D. While the quantile approach is currently limited to one dimension, its insights into smoothing and support growth are foundational. Future research may explore "sliced" MMD methods to apply these 1D results to high-dimensional image generation tasks.
Summary updated