Optimal Transport (OT) is a powerful framework for comparing probability distributions, but it faces two major hurdles: standard OT is computationally expensive (suffering from the "curse of dimensionality"), and it often lacks closed-form solutions. While a closed-form solution exists for Gaussians in the unregularized case (the Bures-Wasserstein distance), the field has lacked one for the entropy-regularized version (the Sinkhorn distance).
Entropy regularization is the industry standard for making OT scalable and smooth. However, because it has been primarily algorithmic, researchers have had no "ground truth" to validate their stochastic solvers against in continuous settings. Furthermore, standard OT requires measures to have the same total mass; "Unbalanced OT" (UOT) relaxes this constraint but adds even more mathematical complexity.
This paper provides a significant theoretical breakthrough by deriving the first closed-form solutions for entropy-regularized optimal transport between Gaussian distributions. By extending these results to the unbalanced case where total mass is not conserved, the authors bridge the gap between elegant theory and scalable, algorithmic practice.
Methodological Approach
The authors tackle the problem by analyzing the Sinkhorn algorithm in the continuous limit. Their key methodological steps include:
- Fixed-Point Analysis: Solving the fixed-point equations that define the optimal dual potentials in the Sinkhorn algorithm.
- Quadratic Ansatz: Proving that if the input measures are Gaussian, the optimal dual potentials are quadratic forms. This crucial step transforms complex functional equations into solvable algebraic matrix equations.
- Matrix Equation Solving: The core of the balanced solution involves solving a Riccati-like matrix equation: $C^2 + \sigma^2 C - AB = 0$.
- Unbalanced Extension: Extending this logic to Unbalanced OT by incorporating Kullback-Leibler divergence penalties on the marginals, which allows for mass creation or destruction.
Key Insights and Theoretical Results
The derivation of these closed forms yields several profound insights into the geometry of regularized transport:
- Differentiability Everywhere: Unlike the standard Bures-Wasserstein distance, which can be non-differentiable when covariance matrices are singular (degenerate), the entropic version is differentiable everywhere. This makes it significantly more stable for gradient-based optimization.
- The Plan is Gaussian: A beautiful theoretical result is that the optimal transport plan between two Gaussians under entropic regularization is itself a (scaled) Gaussian measure.
- The "Mass Destruction" Trade-off: In the unbalanced case, the closed-form formula explicitly quantifies the trade-off between the cost of moving mass and the cost of destroying it. The formula reveals that if means are too far apart, the system prefers to "destroy" mass rather than transport it.
- Entropic Barycenters: The authors also prove that the entropic barycenter (the "average" distribution) of a set of Gaussians is a Gaussian, satisfying a specific fixed-point covariance equation.
The paper formalizes these insights into two main theorems. Theorem 1 (Balanced Case) provides the explicit formula for the regularized transport cost, which resembles a modified Bures distance with an additional log-determinant term accounting for the "blurring" effect of the entropy. Theorem 3 (Unbalanced Case) provides the formula for unbalanced transport, including an exponential term related to the distance between means—showing that transported mass decays as distributions move further apart.
Empirical Validation
As a theoretical paper, the authors validate their findings using synthetic multivariate Gaussians rather than standard machine learning datasets. They generate random Gaussian measures in dimensions $d=5$ and $d=10$, sampling covariance matrices from Wishart distributions to ensure variety across different scales.
These synthetic distributions are used to demonstrate that empirical Sinkhorn estimates (calculated from discrete samples) converge to the theoretical values provided by the new closed-form formulas.
Experiments confirm that as the number of samples $n$ increases, the error between the algorithmic Sinkhorn result and the theoretical closed form vanishes. This convergence follows expected statistical rates across both balanced and unbalanced settings, and across different regularization strengths ($\epsilon$).
Limitations and Future Work
The primary limitation of this work is its reliance on the Gaussian assumption. While Gaussians are ubiquitous in statistics, many real-world data distributions (such as images or text embeddings) are highly non-Gaussian. However, the authors suggest that these closed forms could potentially be extended to Gaussian Mixture Models (GMMs), which would significantly broaden their practical applicability. Ultimately, these formulas serve as a highly valuable standard benchmark for testing new stochastic OT solvers, providing a known "true value" that was previously unavailable to the community.
Summary updated