Debiased Entropic Optimal Transport

27 concepts16 papers1 proposal

Probability distributions are the core currency of modern machine learning, appearing everywhere from generative modeling to shape analysis. To compare these distributions, Optimal Transport (OT) has emerged as a gold standard. By computing the minimum cost required to morph one distribution into another, OT captures the underlying geometry of the data space.

However, exact optimal transport comes with a steep computational price: solving it for $n$ data points requires solving a linear program with $O(n^3 \log n)$ complexity. To scale this up, researchers widely rely on entropic regularization. By adding an entropy penalty to the transport plan, the problem becomes strictly convex and can be solved in $O(n^2)$ time using the elegant, GPU-friendly Sinkhorn algorithm.

But this computational speedup introduces a severe side effect: entropic bias.

The Cost of Speed: Entropic Blurring

To understand entropic bias, we must look at what the entropy penalty actually does. The standard entropic optimal transport objective is formulated as:

$$\text{OT}_\epsilon(\mu, \nu) = \min_{P \in \Pi(\mu, \nu)} \langle P, C \rangle - \epsilon H(P)$$

Here, $C$ is the cost matrix, $P$ is the transport plan, and $H(P)$ is the Shannon entropy of the plan. The regularization parameter $\epsilon$ controls the trade-off between geometric fidelity and smoothness.

When $\epsilon > 0$, the objective actively rewards transport plans with higher entropy—meaning plans that are diffuse, spread out, and blurry. This creates a fundamental mathematical paradox: the regularized distance from a distribution to itself is no longer zero.

$$\text{OT}_\epsilon(\mu, \mu) > 0$$

Because a perfectly deterministic identity mapping has zero entropy, the regularizer forces the transport plan to leak mass to nearby points just to maximize entropy. This violation of the identity of indiscernibles means that $\text{OT}_\epsilon$ is no longer a true metric.

In practice, this entropic bias acts like an artificial force field or a "Gaussian blur" during optimization:

  • In Generative Models (GANs/VAEs): The loss function rewards high-entropy, diffuse states, causing the model to generate blurry samples.
  • In Barycenter Estimation: Averaging multiple distributions results in washed-out, over-smoothed "smudges" instead of sharp, localized features.
  • In Shape Registration: Shapes artificially shrink or lose mass toward their centroids because the metric penalizes the sharp boundaries required to match them exactly.

The Fix: Self-Transport Subtraction

To eliminate this artifact while preserving the computational speed of the Sinkhorn algorithm, a powerful idea emerged: debiasing via self-transport subtraction.

The core mechanism defines a new objective, known as the Sinkhorn Divergence:

$$S_\epsilon(\mu, \nu) = \text{OT}_\epsilon(\mu, \nu) - \frac{1}{2}\text{OT}_\epsilon(\mu, \mu) - \frac{1}{2}\text{OT}_\epsilon(\nu, \nu)$$

The intuition behind this formula is beautifully simple. Think of $\text{OT}_\epsilon(\mu, \mu)$ as a measure of the "auto-correlation" or the intrinsic entropic blur of the distribution $\mu$ itself. By subtracting the self-transport costs of both the source and target distributions from their cross-transport cost, we cancel out the first-order geometric distortions introduced by the entropy term.

As proved by Jean Feydy et al. in "Interpolating between Optimal Transport and MMD using Sinkhorn Divergences", this algebraic correction completely restores the metric properties of the objective. The resulting Sinkhorn Divergence is strictly positive, convex, and—crucially—evaluates to exactly zero if and only if $\mu = \nu$.

Furthermore, Lénaïc Chizat et al. demonstrated in Faster Wasserstein Distance Estimation with the Sinkhorn Divergence that this self-correction term cancels out the primary asymptotic bias relative to the true, unregularized Wasserstein distance. This allows practitioners to use much larger regularization parameters $\epsilon$ (leading to faster convergence) without suffering from the typical geometric distortion.

Restoring Sharpness Across Domains

By ensuring that a distribution's distance to itself is zero, debiased entropic transport changes the behavior of optimization gradients, preventing distributions from collapsing or blurring.

Sharp Generative Modeling

When training generative models, the objective function serves as the guide for the generator. In Learning Generative Models with Sinkhorn Divergences, Aude Genevay et al. showed that shifting from standard entropic OT to the debiased formulation allows models to accurately recover the true data distribution instead of a smoothed approximation.

Similarly, Maziar Sanjabi et al. in On the Convergence and Robustness of Training GANs with Regularized Optimal Transport and Giorgio Patrini et al. in Sinkhorn AutoEncoders leveraged this debiasing effect to eliminate the characteristic blurriness of regularized objectives, enabling the generation of sharp, high-fidelity samples while retaining a smooth loss landscape.

Crisp Barycenters and Shape Registration

Computing the average of multiple distributions under standard entropic transport results in a diffuse cloud. In Debiased Sinkhorn barycenters, Hicham Janati et al. showed that subtracting the self-transport terms completely neutralizes this diffusion effect. This allows the algorithm to compute sharp, clean structural averages at the same high speed as standard Sinkhorn iterations.

Standard Entropic Barycenter:   [ Blurry / Diffuse Smudge ]
Debiased Sinkhorn Barycenter:   [ Sharp / Well-Defined Shape ]

This property is equally vital when aligning complex geometries. Lucas de Lara et al. demonstrated in Diffeomorphic Registration using Sinkhorn Divergences that debiasing prevents shapes from artificially shrinking or losing mass during registration, ensuring that the computed transformations accurately map target boundaries.

Generalizing the Debiasing Principle

The core insight of self-similarity normalization has been successfully extended far beyond standard optimal transport:

Alternative Perspectives and Pitfalls

While self-transport subtraction is highly effective, it is not the only way to eliminate entropic bias, nor is it a universal free lunch.

Iterative Refinement (IPOT)

An alternative to changing the objective function is to change the optimization method itself. In A Fast Proximal Point Method for Computing Exact Wasserstein Distance, Yujia Xie et al. proposed the Inexact Proximal point method for exact Optimal Transport (IPOT). Instead of using a static entropy penalty, IPOT solves a sequence of regularized sub-problems where each step is regularized relative to the previous iteration's solution. This iterative refinement allows the algorithm to converge to the exact, unbiased Wasserstein distance while maintaining the computational robustness and scaling of Sinkhorn methods.

The Map Estimation Pitfall

Practitioners must also distinguish between debiasing a distance value and debiasing a transport map. In Debiaser Beware: Pitfalls of Centering Regularized Transport Maps, Aram-Alexandre Pooladian et al. demonstrated that while self-potential subtraction perfectly corrects distance metrics, applying the same centering technique to estimate the actual point-to-point transport maps can backfire. In finite-sample regimes, the subtraction step can inject significant statistical noise, meaning that standard entropic plans can sometimes outperform debiased ones when mapping actual data samples.

Summary

The development of debiased entropic optimal transport represents a major milestone in geometric machine learning. By recognizing that the smoothing effect of entropic regularization could be quantified and subtracted out via self-transport terms, researchers successfully combined the geometric sensitivity of classical optimal transport with the speed of modern regularized algorithms.

Key Foundational Work

Interpolating between Optimal Transport and MMD using Sinkhorn Divergences (2019) — contributed 2 concept elements to this direction.
Learning Generative Models with Sinkhorn Divergences (2018) — contributed 2 concept elements to this direction.
On the Convergence and Robustness of Training GANs with Regularized Optimal Transport (2018) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_entropic_bias_correction_6574e996,
  title = {entropic bias correction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/entropic-bias-correction/rdr_6574e99655de4daea66c2d01836e5289},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_entropic_bias_correction_6574e996_v1,
  title = {entropic bias correction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/entropic-bias-correction/rdr_6574e99655de4daea66c2d01836e5289/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake