Preserving Homogeneity in Regularized Optimal Transport

18 concepts4 papers1 proposal

Imagine you have two datasets and you compute the optimal transport (OT) plan to align them. Now, imagine you duplicate your data—doubling its total mass—or change the units of your spatial coordinates. Intuitively, the optimal alignment shouldn't fundamentally change; the resulting transport plan should simply scale proportionally. This property is known as homogeneity or scale invariance.

In classical, exact optimal transport, homogeneity is mathematically guaranteed. However, modern machine learning rarely uses classical OT. Instead, we rely on modifications to make OT computationally tractable and robust: entropic regularization (to enable fast, differentiable optimization via the Sinkhorn algorithm) and unbalanced formulations (to handle noise, outliers, and mismatched dataset sizes).

A growing body of research reveals a hidden cost to these modifications: they often break homogeneity. When you scale your data, you don't just scale your results—you fundamentally alter the geometry of the solution.

Why Regularization Breaks Scale Invariance

To understand why homogeneity breaks, consider what happens when we add a standard penalty to an optimization problem. Whether it is an entropic regularization term to smooth the transport plan, or a divergence penalty to allow mass to be created or destroyed in unbalanced OT, these terms typically rely on a fixed, static reference scale.

As Lacombe demonstrates, standard entropic regularization introduces an absolute numerical anchor. When you scale the input measures, the linear transport cost scales linearly, but the entropic penalty does not. This asymmetry shifts the balance between the transport cost and the regularization term. As a result, changing the total mass of your data alters the structural behavior of the optimal transport plan. In complex scenarios like optimal transport with boundaries, this lack of homogeneity can artificially lower the relative cost of discarding mass at a boundary simply because the dataset grew larger.

A similar issue plagues unbalanced transport across different metric spaces, such as matching a 3D shape to a 2D image using the Gromov-Wasserstein distance. As Sejourne et al. observe, relaxing strict mass conservation without adjusting the penalty structure introduces scaling inconsistencies. The alignment cost fails to scale predictably with the total mass of the data, making it incredibly difficult to compare datasets of varying global scales.

Restoring Homogeneity via Relative Constraints

To fix this issue, researchers have shifted from absolute penalties to relative constraints and conic formulations that scale dynamically with the data.

1. Quadratic and Conic Penalties

To preserve homogeneity in unbalanced Gromov-Wasserstein transport, Sejourne et al. developed a formulation that applies divergence penalties to the tensor product of the measures. By utilizing quadratic penalties, the formulation achieves 2-homogeneity. This ensures that the divergence penalty scales at the exact same rate as the mass tensor, maintaining a consistent alignment cost regardless of the global scale of the input data.

2. Relative Mass Budgets

In mini-batch optimal transport, practitioners often use partial transportation to prevent outliers from forcing incorrect pairings. Instead of using an absolute distance penalty to discard outliers, Nguyen et al. introduce a mass budget constraint defined as a relative fraction of the total batch mass.

Because this hyperparameter is defined as a ratio rather than an absolute value, it is inherently scale-invariant. Whether the underlying feature vectors are scaled, transformed, or varied in batch size, the transport behavior remains perfectly stable. This eliminates the need to meticulously retune hyperparameters when data preprocessing scales change.

Absolute Penalty (Inhomogeneous):
Mass Scales Up ──> Penalty Stays Fixed ──> Transport Geometry Distorts

Relative/Conic Constraint (Homogeneous):
Mass Scales Up ──> Penalty Scales Proportionally ──> Transport Geometry Preserved

Renormalization in Numerical Solvers

Even when an optimal transport model is theoretically designed to be scale-invariant, the software implementations running on GPUs can inadvertently reintroduce inhomogeneity.

Renormalized Reference Measures

To resolve the mathematical breakdown caused by entropic regularization, Lacombe proposed Homogeneous Unbalanced Regularized Optimal Transport (HUROT). The core mechanism replaces static reference measures with renormalized reference measures that are dynamically scaled by the total mass of the inputs. This adjustment effectively cancels out the inhomogeneous terms in the dual formulation of the optimization problem.

Normalizing the Stopping Criteria

However, Lacombe also highlights a subtle numerical trap: standard iterative algorithms (like Sinkhorn) use fixed stopping criteria (e.g., terminating when the error is less than a fixed $\epsilon$). If these stopping conditions are not normalized relative to the input mass, the solver will terminate at structurally different qualities of solutions for rescaled versions of the exact same data.

Spatial Rescaling Heuristics

A related practical solution is found in differentiable particle filtering. When using the Sinkhorn algorithm to smoothly resample particles, changes in the dimensionality or magnitude of the state-space can degrade the effectiveness of the entropy parameter. To counteract this, Corenflos et al. implement a spatial rescaling heuristic that normalizes the particles before computing the transport plan. This ensures that the optimization remains robust and the gradients remain stable across varying data scales.

By ensuring that every component of the optimal transport pipeline—from the loss formulation to the numerical convergence criteria—is scale-invariant, these approaches allow deep learning models to utilize optimal transport without fearing that a change in data units or batch size will warp the underlying geometry of the solution.

Key Foundational Work

Improving Mini-batch Optimal Transport via Partial Transportation (2022) — contributed 4 concept elements to this direction.
The Unbalanced Gromov Wasserstein Distance: Conic Formulation and Relaxation (2021) — contributed 3 concept elements to this direction.
Differentiable Particle Filtering via Entropy-Regularized Optimal Transport (2021) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 5 papers
  2. Version 1 · 2026-10-04 · 7 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_numerical_homogeneity_d7e477f1,
  title = {numerical homogeneity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/numerical-homogeneity/rdr_d7e477f116d2452db398414826aa35b4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_numerical_homogeneity_d7e477f1_v2,
  title = {numerical homogeneity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/numerical-homogeneity/rdr_d7e477f116d2452db398414826aa35b4/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake