Debiasing Entropic Optimal Transport with Sinkhorn Divergences

This direction is split into other directions: entropic bias correction. Its last version stays readable and citable here.

Version 1 of 142 papers67 conceptsfrozen 2026-10-04

Optimal Transport (OT) has become a cornerstone of modern machine learning, providing a geometrically sensitive way to compare probability distributions. However, the classic Wasserstein distance is computationally expensive to compute and differentiate. To solve this, researchers led by marco cuturi introduced entropic regularization, which allows the "Sinkhorn algorithm" to compute transport distances at lightning speed.

But this speed comes with a hidden cost: entropic bias. Standard regularized OT is not a true mathematical divergence because the distance from a distribution to itself is not zero. This article explains how "Sinkhorn Divergences" fix this flaw, bridging the gap between the precision of OT and the efficiency of kernel methods.

The Problem: The Blurring Effect of Entropy

In its pure form, Optimal Transport finds the most efficient way to move mass from one distribution to another. When we add an entropic penalty to make this calculation faster, we essentially "blur" the transport plan. Instead of a sharp, one-to-one mapping, the mass is spread out.

As noted by Jean Feydy and Gabriel Peyré in Interpolating between Optimal Transport and MMD using Sinkhorn Divergences, this blurring creates a systematic bias. If you use regularized OT to compare a distribution $\alpha$ to itself, the result is not zero. Instead, the entropy term forces the distribution to look like a "shrunk" or "smeared" version of itself.

In practical tasks like generative modeling or image registration, this bias is disastrous. It acts like a "mean-seeking" force:

  • In Generative Models: The model produces blurry images because the loss function is satisfied by a distribution that collapses toward its own mean Aude Genevay.
  • In Shape Registration: Objects appear to "shrink" or lose mass during the transformation process Lucas de Lara.
  • In Crowd Counting: Predicted density maps become blurry blobs rather than sharp points Hui LIN.

The Solution: The Debiasing Formula

To restore the mathematical properties of a true distance—specifically the identity of indiscernibles ($D(\alpha, \alpha) = 0$)—researchers introduced the Sinkhorn Divergence. The intuition is simple: if the regularized transport cost $OT_\epsilon(\alpha, \beta)$ overestimates the distance by including the "self-blurring" of the distributions, we should simply subtract that self-blurring out.

The formula for the Sinkhorn Divergence $S_\epsilon(\alpha, \beta)$ is: $$S_\epsilon(\alpha, \beta) = OT_\epsilon(\alpha, \beta) - \frac{1}{2}OT_\epsilon(\alpha, \alpha) - \frac{1}{2}OT_\epsilon(\beta, \beta)$$

By subtracting the self-transport terms, we ensure that $S_\epsilon(\alpha, \alpha) = 0$. As Aude Genevay explains in Learning Generative Models with Sinkhorn Divergences, this "centered" formulation allows the loss function to accurately guide a model toward the true data distribution without the artifacts of entropic smoothing.

Bridging Two Worlds: OT and MMD

One of the most elegant properties of the Sinkhorn Divergence is that it acts as a bridge between two major families of distribution distances. This is controlled by the regularization parameter $\epsilon$:

  1. When $\epsilon \to 0$: The divergence converges to the standard Wasserstein distance, which is purely geometric and sensitive to the underlying space.
  2. When $\epsilon \to \infty$: The divergence converges to the Maximum Mean Discrepancy (MMD), a kernel-based distance that is statistically efficient but lacks geometric awareness.

This interpolation, detailed by Jean Feydy in Interpolating between Optimal Transport and MMD using Sinkhorn Divergences, allows practitioners to choose a "sweet spot" that combines the geometric sensitivity of OT with the smooth, differentiable landscape of kernel methods.

Applications Across Domains

Sharp Barycenters

When averaging multiple distributions (finding the "barycenter"), standard entropic OT produces a blurry average. Hicham Janati showed in Debiased Sinkhorn barycenters that by using the debiased divergence, we can recover sharp, high-resolution averages at the same computational speed as the biased version.

Time Series Alignment

In time series analysis, Dynamic Time Warping (DTW) is often used for alignment. However, "soft-DTW" suffers from the same entropic bias as Sinkhorn distances. Mathieu Blondel and colleagues applied the debiasing logic to create the soft-DTW divergence in Differentiable Divergences Between Time Series. This ensures that a time series compared to itself has a distance of zero, preventing the "averaging out" of temporal features during optimization.

Unbalanced and Mini-batch Transport

The debiasing principle has been extended to even more complex scenarios:

Conclusion

The shift from standard Sinkhorn distances to Sinkhorn Divergences represents a move from "fast but flawed" to "fast and theoretically sound." By simply subtracting self-similarity terms, researchers have unlocked a family of loss functions that are differentiable, geometrically aware, and free from the blurring artifacts that once plagued regularized optimal transport. This framework now powers state-of-the-art results in generative modeling, signal processing, and beyond.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_debiasing_entropic_optimal_transport_wit_b6855fe4,
  title = {Debiasing Entropic Optimal Transport with Sinkhorn Divergences},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/debiasing-entropic-optimal-transport-with-sinkhorn-divergences/rdr_b6855fe4adf640e5b8cd179f601f686c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_debiasing_entropic_optimal_transport_wit_b6855fe4_v1,
  title = {Debiasing Entropic Optimal Transport with Sinkhorn Divergences},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/debiasing-entropic-optimal-transport-with-sinkhorn-divergences/rdr_b6855fe4adf640e5b8cd179f601f686c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}