Scale-Invariant and Homogeneous Unbalanced Optimal Transport

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 27 papers20 conceptsfrozen 2026-10-04

Optimal Transport (OT) is often described as the "Earth Mover’s Distance"—a way to calculate the minimum effort required to transform one pile of dirt into another. In machine learning, this "dirt" represents data distributions. However, a subtle problem arises when we apply OT in the wild: if you double the amount of dirt or stretch the ground it sits on, the "optimal" plan for moving it often changes in unpredictable ways.

Recent research has converged on a shared goal: making Optimal Transport scale-invariant and homogeneous. This ensures that the structural relationships the model learns remain consistent regardless of the total mass of the data or the units used to measure it.

The Problem of Forced Matching

In standard Optimal Transport, every unit of mass in the source must be matched to a unit in the target. When training deep learning models, we often compute OT on small "mini-batches" to save memory. As Khai Nguyen, Tung Pham, and their colleagues point out in An Efficient Mini-batch Method via Partial Transportation, this creates "misspecified mappings." Because the mini-batch is just a random slice of the data, the points in the source batch might not actually have a logical counterpart in the target batch. Standard OT forces a match anyway, leading to noisy gradients and poor convergence.

To solve this, Nguyen et al. propose mini-batch Partial Optimal Transport (m-POT). Instead of transporting 100% of the mass, they transport a fixed fraction (e.g., 80%). This fractional constraint is inherently scale-invariant. Whether your batch size is 64 or 1024, the "budget" for transport is relative, allowing the algorithm to ignore outliers and focus on the most relevant matches.

Why Regularization Breaks Homogeneity

To make OT computationally efficient, researchers typically use "entropic regularization" (the Sinkhorn algorithm). This adds a smoothness penalty that makes the problem easier to solve but introduces a new headache: it breaks homogeneity.

In a perfectly homogeneous system, if you scale your input measures by a factor $\lambda$, your transport plan should scale by $\lambda$, and your cost should scale linearly. In An Homogeneous Unbalanced Regularized Optimal Transport Model with Applications to Optimal Transport with Boundary, Théo Lacombe demonstrates that standard entropic penalties destroy this property. This means that changing the "units" or the total mass of your data fundamentally changes the geometry of the solution.

Lacombe introduces HUROT (Homogeneous Unbalanced Regularized OT), which restores this balance by using mass-dependent reference measures. Instead of a static penalty, the regularization term is scaled by the total mass of the inputs. This ensures that the underlying structure of the solution—who gets matched to whom—remains identical even if the data magnitude shifts.

Scaling Across Metric Spaces

The need for scale-invariance extends beyond simple distances to the comparison of entirely different shapes or metric spaces. Thibault Séjourné and François‐Xavier Vialard tackle this in The Unbalanced Gromov Wasserstein Distance: Conic Formulation and Relaxation.

They utilize quadratically-homogeneous divergences. By framing the transport problem such that the cost scales predictably with the total mass, they enable "Unbalanced Gromov-Wasserstein" matching. This allows for robust alignment between datasets that might have different numbers of points or different densities, which is critical for tasks like matching biological datasets or aligning different language embeddings.

Practical Robustness in Dynamic Systems

This theoretical focus on scale has immediate practical benefits in complex models like Particle Filters. In Differentiable Particle Filtering via Entropy-Regularized Optimal Transport, Adrien Corenflos and Arnaud Doucet use OT to resample particles in a way that allows gradients to flow through the model.

To keep the optimization stable across different state-space dimensions, they apply a spatial rescaling heuristic. This ensures that the entropy regularization parameter—which controls how "blurry" the transport plan is—remains effective regardless of whether the particles are spread across a small room or a vast landscape.

Summary of the Intuition

The common thread across these works is that Optimal Transport should care about the shape of the data, not its magnitude.

  1. Partial Transport (Nguyen et al.) uses relative mass budgets to ignore noise in mini-batches.
  2. Homogeneous Regularization (Lacombe) adjusts the penalty terms to ensure that scaling the data doesn't distort the transport plan.
  3. Conic Formulations (Séjourné & Vialard) provide a mathematical framework for comparing distributions with unequal mass.

By enforcing these scale-invariant properties, researchers are transforming OT from a rigid mathematical tool into a flexible, robust component of the modern deep learning toolkit.

Papers in this version

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 5 papers
  2. Version 1 · 2026-10-04 · 7 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_numerical_homogeneity_d7e477f1,
  title = {numerical homogeneity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/numerical-homogeneity/rdr_d7e477f116d2452db398414826aa35b4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_scale_invariant_and_homogeneous_unbalanc_d7e477f1_v1,
  title = {Scale-Invariant and Homogeneous Unbalanced Optimal Transport},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/scale-invariant-and-homogeneous-unbalanced-optimal-transport/rdr_d7e477f116d2452db398414826aa35b4/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}