Properties of Wasserstein Gradient Flows for the Sliced-Wasserstein Distance

Christophe Vauthier, Quentin Mérigot, Anna Korba

Abstract: In this paper, we investigate the properties of the Sliced Wasserstein Distance (SW) when employed as an objective functional. The SW metric has gained significant interest in the optimal transport and machine learning literature, due to its ability to capture intricate geometric properties of probability distributions while remaining computationally tractable, making it a valuable tool for various applications, including generative modeling and domain adaptation. Our study aims to provide a rigorous analysis of the critical points arising from the optimization of the SW objective. By computing explicit perturbations, we establish that stable critical points of SW cannot concentrate on segments. This stability analysis is crucial for understanding the behaviour of optimization algorithms for models trained using the SW objective. Furthermore, we investigate the properties of the SW objective, shedding light on the existence and convergence behavior of critical points. We illustrate our theoretical results through numerical experiments.

Introduction

In machine learning, the task of aligning a source probability distribution with a target distribution is fundamental. While the Wasserstein distance is often considered the gold standard for this task, its high computational cost in high-dimensional spaces has led to the popularity of the Sliced-Wasserstein (SW) distance. SW works by projecting distributions onto random 1D lines ("slices"), where the distance is trivial to compute, and then averaging the results.

Despite its practical success in generative modeling and sampling, SW optimization is inherently non-convex. This raises a critical theoretical question: Can gradient descent get stuck in "bad" local minima? This paper provides a rigorous mathematical analysis of the gradient dynamics of the SW objective, specifically focusing on the stability of its critical points.

Methodology

The authors analyze the optimization landscape through several key theoretical frameworks:

  1. Critical Point Taxonomy: The study defines and relates Eulerian, Wasserstein, and Lagrangian critical points. The focus is placed on Lagrangian critical points, which characterize how the objective responds to perturbations in particle positions.
  2. Barycentric Characterization: A "barycentric" definition of criticality is introduced. A measure is considered a critical point if each particle is already at the "average" position suggested by the aggregate of all 1D slices.
  3. Stability Analysis via Perturbations: To distinguish between stable local minima and unstable saddle points, the authors apply explicit, high-frequency perturbations to candidate critical points.
  4. Discrete-to-Continuous Limits: The paper proves that as the number of particles $N$ approaches infinity, the critical points of the discrete system converge to those of the continuous functional.
  5. Descent Lemma: A specific step size ($\lambda = Nd$) is derived for gradient descent that guarantees a decrease in the SW objective, drawing parallels to the classical Lloyd’s algorithm.

Key Insights and Results

The Instability of Segments

A major contribution of this work is the proof that in 2D, any Lagrangian critical point containing a segment is unstable (Proposition 5.2). While symmetry often allows for the construction of "bad" critical points (e.g., a line segment across the diameter of a circular target), these configurations are shown to be unstable. If particles on such a segment are wiggled even slightly, the SW distance drops significantly, suggesting these points are saddles rather than true local minima.

Numerical Validation

Experiments confirm the theoretical predictions using synthetic distributions such as standard Gaussians, uniform disks, and annuli. As illustrated in Figure 1, perturbing a configuration where particles are concentrated on a central segment leads to an immediate decrease in the $SW_2^2$ distance, confirming the instability of these configurations.

Numerical validation of the instability of segment-based critical points. The top row shows the $SW_2^2$ distance under perturbations, while the bottom row visualizes particle configurations against various target distributions.

Optimization Efficiency

The authors demonstrate that the common practice of using very small step sizes may be unnecessary. By choosing a "barycentric" step size scaled by the number of particles and dimensions ($Nd$), gradient descent can achieve significantly faster initial convergence compared to standard approaches.

Conclusion

This research establishes that for absolutely continuous target measures in 2D, measures supported on segments cannot be stable critical points. This provides a theoretical justification for why gradient-based optimization of the SW objective avoids certain "bad" configurations in practice.

Future work aims to extend these stability proofs to higher dimensions and more complex lower-dimensional structures, such as manifolds. Additionally, while the paper rules out segment-like local minima, a full proof of global convergence remains an open challenge, as other types of stable critical points may yet be identified.

Summary updated

Abstract

In this paper, we investigate the properties of the Sliced Wasserstein Distance (SW) when employed as an objective functional. The SW metric has gained significant interest in the optimal transport and machine learning literature, due to its ability to capture intricate geometric properties of probability distributions while remaining computationally tractable, making it a valuable tool for various applications, including generative modeling and domain adaptation. Our study aims to provide a rigorous analysis of the critical points arising from the optimization of the SW objective. By computing explicit perturbations, we establish that stable critical points of SW cannot concentrate on segments. This stability analysis is crucial for understanding the behaviour of optimization algorithms for models trained using the SW objective. Furthermore, we investigate the properties of the SW objective, shedding light on the existence and convergence behavior of critical points. We illustrate our theoretical results through numerical experiments.
BibTeX
@inproceedings{vauthier2025properties,
  author = {Christophe Vauthier and Quentin Mérigot and Anna Korba},
  title = {{Properties of Wasserstein Gradient Flows for the Sliced-Wasserstein Distance}},
  booktitle = {International Conference on Machine Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=OpLJ5wE1aX},
}
Figure