TL;DR
This paper introduces Riemannian Convex Potential Maps (RCPMs), a novel class of generative models capable of learning complex probability distributions on curved spaces (Riemannian manifolds). By leveraging optimal transport theory and a new type of "discrete c-concave" function, RCPMs offer universal representational power for any compact manifold, outperforming existing methods on synthetic and real-world geological data.
What Problem Does This Paper Address?
Many real-world datasets don't live on simple, flat Euclidean spaces. Instead, they reside on curved surfaces like spheres, tori, or more complex manifolds, common in fields like physics and geology. Existing generative models, designed for Euclidean data, struggle with these curved spaces. They either incorrectly assign probability mass to impossible regions or require significant effort to "squeeze" data into feasible areas. Furthermore, these models often lack a deep understanding of the underlying manifold's geometry, limiting their representational capacity and efficiency. This paper aims to bridge this gap by developing a universal generative model that inherently respects and leverages the geometry of Riemannian manifolds.
Methods
The core innovation of this paper is the Riemannian Convex Potential Map (RCPM). RCPMs are built upon the theory of Optimal Transport (OT) on Riemannian manifolds, specifically generalizing convex potential flows from the Euclidean setting.
Here's a breakdown of the key methodological components:
- Riemannian Optimal Transport: Instead of standard convex functions, RCPMs use c-concave functions. These are generalizations of concave functions adapted to the geometry of a manifold, where the "distance" is the intrinsic geodesic distance.
- Discrete c-concave Functions: A significant challenge is that c-concave functions are intricate and hard to characterize. The authors introduce discrete c-concave functions as an approximation. These are formed by taking the minimum of a finite number of simple functions based on squared intrinsic distances to a set of discrete points. This construction is shown to be dense in the space of all c-concave functions, meaning they can approximate any c-concave function arbitrarily well.
- Exponential Map Flows: The actual generative "flow" is constructed using the Riemannian exponential map. This map takes a tangent vector at a point on the manifold and "transports" it along a geodesic to a new point. In RCPMs, the tangent vector is derived from the gradient of the discrete c-concave potential.
- Universality: The paper proves that RCPMs, even with a single layer, are universally expressive. This means they can approximate any optimal transport map between an absolutely continuous source distribution and any target distribution on a compact Riemannian manifold.
- Architecture: RCPMs are built as a composition of these discrete c-concave potential blocks. They also explore multi-layer blocks and smoothing techniques (soft-min) to improve optimization and ensure the resulting maps are diffeomorphisms.
Datasets
The paper evaluates RCPMs on several datasets and scenarios:
- Synthetic Datasets:
- 4-modal distribution on a sphere (S²): Used for density estimation and comparison with existing methods.
- Checkerboard distribution on a sphere (S²): A more challenging dataset for density estimation.
- 3-modal distribution on a torus (T² = S¹ × S¹): Demonstrates RCPM's applicability to product manifolds.
- Real-world Geological Data:
- Continental drift data: Used for mapping estimation (tracking ground mass movement over time) and transport geodesics. The data consists of ground mass distributions on Earth from 90 million years ago and the present day.
- Earth's land mass distribution: Used for density estimation on a sphere.
Key Insights and Ideas
- Geometry is Key: The paper strongly emphasizes that for non-Euclidean data, explicitly incorporating manifold geometry is crucial for effective generative modeling. RCPMs achieve this by using Riemannian OT and exponential maps.
- Discrete Approximations for Intricate Functions: The introduction of "discrete c-concave functions" is a clever way to make the theoretical framework of Riemannian OT computationally tractable and universally applicable. This is a significant theoretical contribution.
- Universal Representational Power: The proof of universality for RCPMs is a major finding. It suggests that these models are not limited by the complexity of the target distribution on any compact manifold, a significant advantage over many existing methods.
- Bridging Theory and Practice: The paper successfully translates complex theoretical concepts from Riemannian geometry and optimal transport into a practical, implementable generative model.
Key Results
- Superior Density Estimation: On the 4-modal sphere dataset, RCPM achieved a KL divergence of 0.003, which is almost an order of magnitude lower than the best previous method (0.05). This highlights its superior expressive power.
- Comparable Training Speed: RCPM's training time per iteration was comparable to existing methods like Rezende et al. (2020), while achieving significantly better KL and ESS (Effective Sample Size).
- Successful Modeling on Spheres and Tori: RCPMs accurately modeled complex distributions on both synthetic spheres and tori.
- Real-World Application in Geology: The model successfully demonstrated mapping estimation for continental drift, accurately tracking the movement of ground mass over millions of years. It also generated realistic transport geodesics illustrating tectonic plate movements.
- Universality Demonstrated: The theoretical proof of universality suggests RCPMs can handle arbitrary target distributions on compact manifolds.
Limitations and Future Work
- Diffeomorphism Guarantee: While the proposed RCPM construction is based on OT, the paper notes that the resulting maps are not guaranteed to be diffeomorphisms (smooth, invertible maps with smooth inverses). They propose smoothing the "min" operation with a "soft-min" to achieve this, but acknowledge that preserving c-concavity with this smoothing is an open question.
- Manifold Assumptions: The current framework relies on the exponential map and intrinsic distance being known and computable. Extending this to more general manifolds where these are not readily available is a future direction.
- Generalizing Barycenters: The authors suggest that RCPMs could be used to generalize the concept of barycenters (means) of probability distributions from Euclidean spaces to Riemannian manifolds.
- Other Optimization Methods: Exploring alternative optimization strategies beyond standard flow-based training, such as Wasserstein loss or semi-discrete transport methods, is suggested.
- Manifolds with Signed Distance Functions: Future work includes training flows on manifolds defined with signed distance functions.
Summary updated