TL;DR
This paper introduces a novel generalization of the Wasserstein barycenter, allowing probability measures to live on different, lower-dimensional subspaces of a larger ambient space. This framework enables the reconstruction of multidimensional distributions from their projections and offers a powerful tool for tasks like image reconstruction and data aggregation.
What Problem Does This Paper Address?
The classical Wasserstein barycenter finds a "mean" probability distribution that minimizes the average squared Wasserstein distance to a set of given distributions. However, this assumes all distributions live in the same space. Many real-world problems involve data that are naturally represented as projections or measurements in lower-dimensional subspaces of a higher-dimensional space. For instance, reconstructing a 3D object from 2D images or inferring a high-dimensional model from lower-dimensional patch-based representations. The challenge is to find a consensus distribution in the high-dimensional space that is consistent with these lower-dimensional projections, even if these projections are noisy or imperfect. This paper tackles this by generalizing the Wasserstein barycenter to handle probability measures residing in different subspaces.
Methods
The core of the paper is the introduction of the Generalized Wasserstein Barycenter (GWB). The problem is formulated as minimizing a sum of Wasserstein distances between each target measure and the "projected" barycenter. The authors establish the existence of solutions and relate the GWB problem to a multi-marginal optimal transport problem in a higher-dimensional space.
Key methodological contributions include:
- Technical Reduction: A crucial step involves transforming the problem into a standard Wasserstein barycenter problem by mapping the original measures to a common space using a matrix derived from the projection operators. This allows leveraging existing tools.
- Dual Formulation: A dual problem is derived, providing theoretical insights and a potential avenue for computation.
- Gaussian Solution: For Gaussian distributions, the paper shows that a Gaussian solution exists and provides explicit formulas for its mean and covariance matrix. This is achieved by solving a fixed-point equation for the covariance.
- Numerical Computation: Strategies for computing the GWB for discrete distributions are discussed, including using linear programming for exact solutions and the Sinkhorn algorithm for approximations.
Datasets
Not discussed in the paper. The paper focuses on theoretical development and illustrates its concepts with synthetic examples and visualizations rather than using specific datasets.
Key Insights and Ideas
- Bridging Subspaces: The most significant insight is the ability to define a meaningful "average" distribution even when the input distributions live in different dimensional spaces. This is achieved by projecting them into a common ambient space.
- Reconstruction from Projections: The GWB provides a principled way to reconstruct a high-dimensional distribution from its lower-dimensional projections, which is a fundamental problem in many scientific fields.
- Compromise Between Disagreeing Marginals: The GWB naturally finds a compromise distribution that best aligns with all projections, even if those projections are not perfectly consistent.
- Gaussian Solutions are Gaussian: A key theoretical result is that if the input distributions are Gaussian, the generalized barycenter is also Gaussian, with its parameters explicitly computable.
Key Results
- Existence and Uniqueness: The paper proves the existence of solutions for the GWB problem. Uniqueness is generally lost, but the projections of the barycenter are shown to be unique under certain conditions (e.g., if input measures have densities).
- Gaussian Solution Formulas: Explicit formulas are derived for the mean and covariance matrix of the Gaussian GWB.
- Connection to Multi-Marginal OT: The GWB problem is shown to be equivalent to a multi-marginal optimal transport problem in a higher-dimensional space, enabling the use of existing algorithms.
- Numerical Methods: Practical algorithms for computing GWB for discrete and Gaussian distributions are presented and illustrated with examples.
Limitations and Future Work
- Uniqueness: The paper acknowledges that the generalized barycenter itself is often not unique, although its projections might be. This is a common characteristic of barycenter problems.
- Computational Cost: While numerical methods are proposed, solving the multi-marginal optimal transport problem can still be computationally intensive, especially for a large number of marginals or high-dimensional spaces. The paper mentions that methods optimizing only mass locations are faster than full multi-marginal strategies.
- Gaussian Mixtures: The paper extends the framework to Gaussian mixtures, but the classical optimal transport barycenter of Gaussian mixtures does not necessarily remain a Gaussian mixture. The proposed MW2 distance aims to address this.
- Future Work: The authors suggest exploring the dual formulation further and applying the GWB to more complex real-world scenarios, particularly in imaging and data aggregation. The robustness of reconstruction with noisy projections is also highlighted as an area for further investigation.
Summary updated