Entropic Optimal Transport between Unbalanced Gaussian Measures has a Closed Form

Hicham Janati, Boris Muzellec, Gabriel Peyré, Marco Cuturi

Optimal Transport (OT) is a powerful framework for comparing probability distributions, but it faces two major hurdles: standard OT is computationally expensive (suffering from the "curse of dimensionality"), and it often lacks closed-form solutions. While a closed-form solution exists for Gaussians in the unregularized case (the Bures-Wasserstein distance), the field has lacked one for the entropy-regularized version (the Sinkhorn distance).

Entropy regularization is the industry standard for making OT scalable and smooth. However, because it has been primarily algorithmic, researchers have had no "ground truth" to validate their stochastic solvers against in continuous settings. Furthermore, standard OT requires measures to have the same total mass; "Unbalanced OT" (UOT) relaxes this constraint but adds even more mathematical complexity.

This paper provides a significant theoretical breakthrough by deriving the first closed-form solutions for entropy-regularized optimal transport between Gaussian distributions. By extending these results to the unbalanced case where total mass is not conserved, the authors bridge the gap between elegant theory and scalable, algorithmic practice.

Methodological Approach

The authors tackle the problem by analyzing the Sinkhorn algorithm in the continuous limit. Their key methodological steps include:

  • Fixed-Point Analysis: Solving the fixed-point equations that define the optimal dual potentials in the Sinkhorn algorithm.
  • Quadratic Ansatz: Proving that if the input measures are Gaussian, the optimal dual potentials are quadratic forms. This crucial step transforms complex functional equations into solvable algebraic matrix equations.
  • Matrix Equation Solving: The core of the balanced solution involves solving a Riccati-like matrix equation: $C^2 + \sigma^2 C - AB = 0$.
  • Unbalanced Extension: Extending this logic to Unbalanced OT by incorporating Kullback-Leibler divergence penalties on the marginals, which allows for mass creation or destruction.

Key Insights and Theoretical Results

The derivation of these closed forms yields several profound insights into the geometry of regularized transport:

  • Differentiability Everywhere: Unlike the standard Bures-Wasserstein distance, which can be non-differentiable when covariance matrices are singular (degenerate), the entropic version is differentiable everywhere. This makes it significantly more stable for gradient-based optimization.
  • The Plan is Gaussian: A beautiful theoretical result is that the optimal transport plan between two Gaussians under entropic regularization is itself a (scaled) Gaussian measure.
  • The "Mass Destruction" Trade-off: In the unbalanced case, the closed-form formula explicitly quantifies the trade-off between the cost of moving mass and the cost of destroying it. The formula reveals that if means are too far apart, the system prefers to "destroy" mass rather than transport it.
  • Entropic Barycenters: The authors also prove that the entropic barycenter (the "average" distribution) of a set of Gaussians is a Gaussian, satisfying a specific fixed-point covariance equation.

The paper formalizes these insights into two main theorems. Theorem 1 (Balanced Case) provides the explicit formula for the regularized transport cost, which resembles a modified Bures distance with an additional log-determinant term accounting for the "blurring" effect of the entropy. Theorem 3 (Unbalanced Case) provides the formula for unbalanced transport, including an exponential term related to the distance between means—showing that transported mass decays as distributions move further apart.

Empirical Validation

As a theoretical paper, the authors validate their findings using synthetic multivariate Gaussians rather than standard machine learning datasets. They generate random Gaussian measures in dimensions $d=5$ and $d=10$, sampling covariance matrices from Wishart distributions to ensure variety across different scales.

These synthetic distributions are used to demonstrate that empirical Sinkhorn estimates (calculated from discrete samples) converge to the theoretical values provided by the new closed-form formulas.

Convergence of empirical Sinkhorn estimates to theoretical closed forms

Experiments confirm that as the number of samples $n$ increases, the error between the algorithmic Sinkhorn result and the theoretical closed form vanishes. This convergence follows expected statistical rates across both balanced and unbalanced settings, and across different regularization strengths ($\epsilon$).

Limitations and Future Work

The primary limitation of this work is its reliance on the Gaussian assumption. While Gaussians are ubiquitous in statistics, many real-world data distributions (such as images or text embeddings) are highly non-Gaussian. However, the authors suggest that these closed forms could potentially be extended to Gaussian Mixture Models (GMMs), which would significantly broaden their practical applicability. Ultimately, these formulas serve as a highly valuable standard benchmark for testing new stochastic OT solvers, providing a known "true value" that was previously unavailable to the community.

Summary updated

Abstract

Although optimal transport (OT) problems admit closed form solutions in a very few notable cases, e.g. in 1D or between Gaussians, these closed forms have proved extremely fecund for practitioners to define tools inspired from the OT geometry. On the other hand, the numerical resolution of OT problems using entropic regularization has given rise to many applications, but because there are no known closed-form solutions for entropic regularized OT problems, these approaches are mostly algorithmic, not informed by elegant closed forms. In this paper, we propose to fill the void at the intersection between these two schools of thought in OT by proving that the entropy-regularized optimal transport problem between two Gaussian measures admits a closed form. Contrary to the unregularized case, for which the explicit form is given by the Wasserstein-Bures distance, the closed form we obtain is differentiable everywhere, even for Gaussians with degenerate covariance matrices. We obtain this closed form solution by solving the fixed-point equation behind Sinkhorn's algorithm, the default method for computing entropic regularized OT. Remarkably, this approach extends to the generalized unbalanced case --- where Gaussian measures are scaled by positive constants. This extension leads to a closed form expression for unbalanced Gaussians as well, and highlights the mass transportation / destruction trade-off seen in unbalanced optimal transport. Moreover, in both settings, we show that the optimal transportation plans are (scaled) Gaussians and provide analytical formulas of their parameters. These formulas constitute the first non-trivial closed forms for entropy-regularized optimal transport, thus providing a ground truth for the analysis of entropic OT and Sinkhorn's algorithm.
BibTeX
@inproceedings{janati2020entropic,
  author = {Hicham Janati and Boris Muzellec and Gabriel Peyré and Marco Cuturi},
  title = {{Entropic Optimal Transport between Unbalanced Gaussian Measures has a Closed Form}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=SU5TvggfgG},
}
Figure