Entropic Optimal Transport between Unbalanced Gaussian Measures has a Closed Form

Hicham Janati, Boris Muzellec, Gabriel Peyré, Marco Cuturi

Abstract

Although optimal transport (OT) problems admit closed form solutions in a very few notable cases, e.g. in 1D or between Gaussians, these closed forms have proved extremely fecund for practitioners to define tools inspired from the OT geometry. On the other hand, the numerical resolution of OT problems using entropic regularization has given rise to many applications, but because there are no known closed-form solutions for entropic regularized OT problems, these approaches are mostly algorithmic, not informed by elegant closed forms. In this paper, we propose to fill the void at the intersection between these two schools of thought in OT by proving that the entropy-regularized optimal transport problem between two Gaussian measures admits a closed form. Contrary to the unregularized case, for which the explicit form is given by the Wasserstein-Bures distance, the closed form we obtain is differentiable everywhere, even for Gaussians with degenerate covariance matrices. We obtain this closed form solution by solving the fixed-point equation behind Sinkhorn's algorithm, the default method for computing entropic regularized OT. Remarkably, this approach extends to the generalized unbalanced case -- where Gaussian measures are scaled by positive constants. This extension leads to a closed form expression for unbalanced Gaussians as well, and highlights the mass transportation / destruction trade-off seen in unbalanced optimal transport. Moreover, in both settings, we show that the optimal transportation plans are (scaled) Gaussians and provide analytical formulas of their parameters. These formulas constitute the first non-trivial closed forms for entropy-regularized optimal transport, thus providing a ground truth for the analysis of entropic OT and Sinkhorn's algorithm.

1. Introduction

Optimal Transport (OT) has emerged as a cornerstone framework for comparing probability distributions in machine learning and statistics. However, standard OT faces two significant practical challenges: high computational complexity and a lack of smoothness (differentiability) when dealing with degenerate or "thin" distributions. To mitigate these issues, researchers frequently employ entropic regularization, which allows for efficient computation via the Sinkhorn algorithm.

While the unregularized OT problem between Gaussian measures has a well-known closed-form solution—the Wasserstein-Bures metric—the entropic variant has historically remained an algorithmic "black box." This work provides the first analytical closed-form solutions for both balanced and unbalanced entropic OT between Gaussian measures, offering a differentiable and robust alternative to classical metrics.

2. Theoretical Framework and Methodology

The core contribution of this work is the derivation of explicit formulas for the entropic OT cost by analyzing the fixed-point equations of the Sinkhorn algorithm. The methodology follows several key steps:

  • Dual Potential Analysis: The authors demonstrate that for Gaussian marginals, the dual potentials (the variables optimized by the Sinkhorn algorithm) take a quadratic form.
  • Fixed-Point Resolution: By substituting these quadratic forms into the Sinkhorn iterations, the problem reduces to solving a matrix algebraic equation—specifically, a variation of a Riccati equation—that characterizes the optimal transport plan.
  • Unbalanced Extension: The derivation is extended to the unbalanced setting, which incorporates Kullback-Leibler (KL) penalties. This allows for mass creation or destruction, a necessary feature when comparing distributions with unequal total weights.
  • Debiasing via Sinkhorn Divergence: To ensure the metric is properly grounded (where the distance between a distribution and itself is zero), the authors utilize Sinkhorn divergences to remove entropic bias.

3. Numerical Validation

To validate the theoretical derivations, the authors conducted experiments using synthetic multivariate Gaussian measures in dimensions $d=5$ and $d=10$. Covariance structures were generated using Wishart distributions. The empirical Sinkhorn algorithm, run on finite samples, was compared against the derived closed-form "ground truth."

Convergence of empirical Sinkhorn estimates to the derived closed-form solutions (dashed lines) for various dimensions and regularization parameters.

As illustrated in the figure above, the empirical estimates converge perfectly to the theoretical values as the number of samples increases. This confirms that the derived formulas provide an exact analytical description of the entropic OT cost.

4. Key Insights and Properties

The analytical solutions reveal several critical properties of entropic OT:

  • Global Differentiability: Unlike the standard Wasserstein-Bures metric, the entropic OT cost is differentiable everywhere, even for singular covariance matrices. This makes it significantly more stable for gradient-based optimization in machine learning.
  • Transport-Destruction Trade-off: In the unbalanced case, the formulas explicitly quantify the balance between moving mass and destroying/re-creating it. As the distance between means increases, the amount of transported mass mathematically decays toward zero, favoring local mass destruction.
  • Gaussian Preservation: The optimal transport plan between two Gaussians is itself a (scaled) Gaussian. This structural preservation mirrors the unregularized case and simplifies downstream theoretical analysis.

5. Conclusion and Future Work

By providing the first non-trivial closed forms for entropy-regularized OT, this work establishes a vital benchmark for the field. These formulas allow for a more rigorous analysis of the sample complexity of entropic OT and provide a robust replacement for the Bures metric in applications such as generative modeling (GANs) and domain adaptation. Future research may leverage these results to develop more efficient stochastic Sinkhorn variants and explore barycenter computations in the entropic regime.

Summary updated

Abstract

Although optimal transport (OT) problems admit closed form solutions in a very few notable cases, e.g. in 1D or between Gaussians, these closed forms have proved extremely fecund for practitioners to define tools inspired from the OT geometry. On the other hand, the numerical resolution of OT problems using entropic regularization has given rise to many applications, but because there are no known closed-form solutions for entropic regularized OT problems, these approaches are mostly algorithmic, not informed by elegant closed forms. In this paper, we propose to fill the void at the intersection between these two schools of thought in OT by proving that the entropy-regularized optimal transport problem between two Gaussian measures admits a closed form. Contrary to the unregularized case, for which the explicit form is given by the Wasserstein-Bures distance, the closed form we obtain is differentiable everywhere, even for Gaussians with degenerate covariance matrices. We obtain this closed form solution by solving the fixed-point equation behind Sinkhorn's algorithm, the default method for computing entropic regularized OT. Remarkably, this approach extends to the generalized unbalanced case -- where Gaussian measures are scaled by positive constants. This extension leads to a closed form expression for unbalanced Gaussians as well, and highlights the mass transportation / destruction trade-off seen in unbalanced optimal transport. Moreover, in both settings, we show that the optimal transportation plans are (scaled) Gaussians and provide analytical formulas of their parameters. These formulas constitute the first non-trivial closed forms for entropy-regularized optimal transport, thus providing a ground truth for the analysis of entropic OT and Sinkhorn's algorithm.
BibTeX
@inproceedings{janati2020entropic,
  author = {Hicham Janati and Boris Muzellec and Gabriel Peyré and Marco Cuturi},
  title = {{Entropic Optimal Transport between Unbalanced Gaussian Measures has a Closed Form}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=SU5TvggfgG},
}
Figure