Imagine you have two simple probability distributions on a line: one sharp peak centered at $-2$, and another sharp peak centered at $+2$. If you take their standard arithmetic (Euclidean) average, what do you get? You get a bimodal distribution with two smaller peaks at $-2$ and $+2$, or a flattened, blurry blob centered at $0$. The probability density at $0$ increases, even though neither of the original distributions had any mass there.
This is the fundamental flaw of treating probability distributions as simple coordinate vectors in a Euclidean space. When we average them point-wise, we measure distance vertically—looking only at the difference in probability density at a specific coordinate. This vertical averaging completely ignores the horizontal geometry of the underlying space, resulting in a "blurring" effect where distinct patterns and structural features are washed away into a generic data cloud.
To preserve the functional shape, sharpness, and topology of our data, we need a way to average distributions geometrically rather than arithmetically.
The Geometric Alternative: Moving Mass
Instead of comparing densities vertically, Optimal Transport (OT) shifts our perspective from "averaging values" to "moving mass." It measures the horizontal distance and effort required to physically transport the mass of one distribution to match another. The minimum cost to achieve this transformation is the Wasserstein distance.
As explained by Marco Cuturi in Computational Optimal Transport, geometric distribution averaging computes a central representative of multiple probability measures by minimizing the weighted sum of these transport costs. This central representative is known as the Wasserstein barycenter (or the Fréchet mean within a Wasserstein metric space).
Returning to our original example: the Wasserstein barycenter of the two peaks at $-2$ and $+2$ is a single, sharp peak centered at $0$. Instead of melting into a flat puddle, the mass from both sides smoothly travels along the underlying manifold to meet in the middle. The structural shape, modality, and sharpness of the distribution are perfectly preserved.
As Mark Bun et al. show in Differentially Private Wasserstein Barycenters, minimizing the sum of squared Optimal Transport distances allows an aggregate distribution to retain spatial structure and topological features that are completely lost in coordinate-wise averaging.
Why It Works Across Domains
This ability to preserve structural integrity makes Wasserstein barycenters an incredibly powerful tool across machine learning, where aggregating diverse distributions is a core challenge.
Neuroimaging and Signal Processing
In neuroimaging, individual brain activation maps (like fMRI or MEG) vary slightly in anatomy from person to person. Standard voxel-wise averaging blurs these signals, causing severe signal loss. Marco Cuturi et al. in Fast Optimal Transport Averaging of Neuroimaging Data and Hicham Janati et al. in Multi-subject MEG/EEG source imaging with sparse multi-task regression demonstrate that Wasserstein barycenters preserve the focal intensity and sharpness of group-level neural signals by explicitly accounting for spatial shifts and the underlying geometry of the brain.
Distributed Bayes and Federated Learning
When scaling Bayesian inference to massive datasets using a divide-and-conquer strategy, arithmetic averaging of subset posteriors fails to represent true uncertainty. Sanvesh Srivastava et al. introduced the WASP framework (WASP: Scalable Bayes via barycenters of subset posteriors), proving that combining subset posteriors via Wasserstein barycenters maintains the characteristic shape and geometric features of the true distribution.
Similarly, in personalized federated learning, Ting Wei et al. (Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregation) utilize server-side barycentric aggregation to merge client models. This treats local parameter distributions as physical masses on a manifold, preventing the loss of structural information in highly heterogeneous environments.
Multi-Modal Fusion and Representation Learning
Integrating data from diverse modalities (such as text, images, and graphs) often suffers from spatial heterogeneity. Peijie Qiu et al. (Multimodal Variational Autoencoder: A Barycentric View) demonstrate that computing a latent barycenter using the 2-Wasserstein distance respects the underlying manifold geometry, allowing for natural interpolation between modalities.
For knowledge graphs, Zongsheng Cao et al. (OTKGE: Multi-modal Knowledge Graph Embeddings via Optimal Transport) use Wasserstein barycenters to fuse multi-modal features. This geometric average maintains the statistical spread and discriminative power of the input data, avoiding the information collapse typical of simple vector concatenation or averaging.
Domain Adaptation
In multi-source domain adaptation, a model must consolidate knowledge from multiple source domains to generalize to a target domain. Ievgen Redko et al. (Theoretical Analysis of Domain Adaptation with Optimal Transport) provided the theoretical foundation for this, proving that constructing an intermediate representative distribution via the Wasserstein barycenter of the source domains offers a mathematically grounded strategy for minimizing target domain error.
Scaling the Computation
Historically, the primary obstacle to using Wasserstein barycenters was computational complexity, as solving exact optimal transport linear programs is notoriously slow. However, the field has undergone a scalability revolution.
Beyond the popular entropic regularization methods (like the Sinkhorn algorithm), recent frameworks allow for the aggregation of continuous, high-dimensional distributions without needing discrete point-cloud approximations. For example, Jinjin Chi et al. (Variational Wasserstein Barycenters with C-cyclical Monotonicity Regularization) introduce variational methods to preserve scalability in high dimensions.
Additionally, Maxence Noble et al. (Tree-Based Diffusion Schrödinger Bridge with Applications to Wasserstein Barycenters) frame barycenter computation as a multi-marginal problem on a star-shaped tree topology, leveraging diffusion-based generative models to synchronize transport paths toward a central hub. These algorithmic breakthroughs have transitioned Wasserstein barycenters from an elegant theoretical concept into a highly scalable tool for modern machine learning pipelines.
Key Foundational Work
WASP: Scalable Bayes via barycenters of subset posteriors (2015) — contributed 2 concept elements to this direction.
Distribution’s template estimate with Wasserstein metrics (2015) — contributed 1 concept elements to this direction.
Fast Optimal Transport Averaging of Neuroimaging Data (2015) — contributed 1 concept elements to this direction.