Abstract
This paper introduces a novel deep learning framework designed to compute geodesics—the optimal paths between probability distributions—under the spherical Wasserstein-Fisher-Rao (WFR) metric. Unlike traditional Optimal Transport (OT), which is constrained by mass conservation, the WFR metric allows for the creation and destruction of mass. By parameterizing the transformation as a fluid flow with a source term, the proposed UOT-gen framework enables efficient weighted sample generation and density estimation, particularly in complex scenarios such as sequential Bayesian inference.
Introduction
In machine learning and statistics, measuring the distance between probability distributions and finding optimal transformations between them are fundamental tasks. For years, Optimal Transport (OT) and the associated Wasserstein distance have served as the gold standard. However, standard OT operates under a rigid constraint: the total mass must be conserved. In physical terms, you can move "dirt" from one location to another, but you cannot create or destroy it.
This constraint becomes problematic in "unbalanced" problems where distributions have different total masses or where it is more computationally efficient to let mass grow or shrink rather than travel long distances. The Wasserstein-Fisher-Rao (WFR) metric was developed to address this by integrating a "transport + growth" trade-off. Despite its theoretical elegance, computing WFR geodesics—the shortest paths of evolution—is mathematically daunting. Existing flow-based models often struggle to handle weighted data without expensive resampling. This paper presents a deep learning solution to bridge this gap.
Methodology: The UOT-gen Framework
The authors propose UOT-gen (Unbalanced OT-generation), a framework that treats the transformation between distributions as a dynamic fluid flow. The core of the method relies on several key technical innovations:
1. Dynamic Formulation
Following the Benamou-Brenier approach, the model views the transformation as a continuous process over time. It solves for two primary components: a velocity field ($v$), which dictates how particles move, and a source term ($g$), which describes how the weight (mass) of those particles changes.
2. Neural Potential Functions
Rather than learning the velocity and source terms directly, the framework parameterizes a potential function $\Phi$ using a ResNet architecture. The velocity and weight changes are then derived from the gradients of this potential, ensuring the resulting flow remains physically and mathematically consistent with the WFR metric.
3. Inverse KL Divergence
To ensure the starting distribution accurately reaches the target, the authors introduce a Kullback-Leibler (KL) divergence term based on the inverse map. This is a critical design choice: it allows the model to handle weight changes during training without requiring explicit knowledge of the starting distribution's density, making it highly flexible for generative tasks.
4. Velocity Regularization
A specialized regularization term is introduced to penalize deviations from the optimal relationship between particle velocity and weight change. This encourages the neural network to converge specifically on the true geodesic path rather than any arbitrary transformation.
Experimental Evaluation
The framework was evaluated across several synthetic and practical benchmarks:
- 1D Gaussian Mixtures: These tests visualized the competition between the "birth-death" effect (weight change) and transportation.
- 2D Toy Problems: Using standard benchmarks like the "8 Gaussians" and "Moons" datasets, the model demonstrated its ability to reconstruct complex, disjoint supports starting from a simple normal distribution.
- Bayesian Posterior Estimation: The authors applied UOT-gen to a 1D Bernoulli equation problem. The goal was to estimate an initial state from noisy sequential observations. The model successfully ingested weighted samples to learn the posterior distribution, bypassing the need for traditional Markov Chain Monte Carlo (MCMC) methods.
Key Insights
The "Lazy Particle" Effect
A significant finding is that under the WFR metric, particles can exhibit "lazy" behavior. If the cost of transporting a particle across a large distance exceeds the cost of destroying it at the source and regrowing it at the destination, the WFR metric will favor the latter. This provides a more flexible and often more natural path for distribution morphing than standard OT.
The $\alpha$ Parameter as a Physics Controller
The hyperparameter $\alpha$ serves as a dial to control the model's behavior. A large $\alpha$ forces the model toward standard Wasserstein transport (moving mass), while a small $\alpha$ emphasizes the Fisher-Rao component (changing weights in place). This allows practitioners to tune the "physics" of the transformation based on the specific needs of the dataset.
Application to Bayesian Inference
UOT-gen is naturally suited for Bayesian tasks. In these scenarios, we often deal with samples from a prior that are weighted by a likelihood. UOT-gen can process these weighted samples directly to learn the posterior, offering a promising path for online Bayesian updates as new data arrives.
Conclusion and Future Work
UOT-gen provides a robust deep learning framework for computing WFR geodesics, successfully capturing the theoretical properties of unbalanced optimal transport. While the model performs well on 2D benchmarks and Bayesian tasks, scaling to very high-dimensional density estimation remains an area for future research. The authors suggest that this framework could be particularly impactful in fields like drug molecule design, where the ability to balance structural preservation (transport) with property modification (weight change) is highly valuable.
Summary updated