Hyperbolic Sliced-Wasserstein via Geodesic and Horospherical Projections

Clément Bonet, Laetitia Chapel, Lucas Drumetz, Nicolas Courty

Abstract

Hyperbolic space embeddings have been shown beneficial for many learning tasks where data have an underlying hierarchical structure. Consequently, many machine learning tools were extended to such spaces, but only few discrepancies to compare probability distributions defined over those spaces exist. Among the possible candidates, optimal transport distances are well defined on such Riemannian manifolds and enjoy strong theoretical properties, but suffer from high computational cost. On Euclidean spaces, sliced-Wasserstein distances, which leverage a closed-form solution of the Wasserstein distance in one dimension, are more computationally efficient, but are not readily available on hyperbolic spaces. In this work, we propose to derive novel hyperbolic sliced-Wasserstein discrepancies. These constructions use projections on the underlying geodesics either along horospheres or geodesics. We study and compare them on different tasks where hyperbolic representations are relevant, such as sampling or image classification.

Introduction

In modern machine learning, the choice of embedding space is crucial for capturing the intrinsic structure of data. While Euclidean space is the standard, hyperbolic space has emerged as a powerful alternative for representing hierarchical data, such as taxonomies, social networks, and biological trees. The exponential growth of volume in hyperbolic space naturally accommodates the branching structure of trees, allowing for low-distortion embeddings.

However, a significant challenge arises when comparing probability distributions in these non-Euclidean spaces. The Wasserstein distance, derived from Optimal Transport (OT), is a robust metric but is computationally expensive, scaling super-cubically with the number of samples. In Euclidean settings, the Sliced-Wasserstein (SW) distance offers a scalable alternative by projecting high-dimensional distributions onto one-dimensional lines. Extending this "slicing" technique to hyperbolic space requires new projection operators that respect the manifold's curvature.

Methodology: Hyperbolic Slicing

The core contribution of this work is the definition of two novel slicing strategies for hyperbolic space, specifically designed for the Poincaré ball and Lorentz (hyperboloid) models. Instead of projecting onto Euclidean lines, the proposed methods project data onto geodesics—the hyperbolic equivalent of straight lines.

Geodesic Projection (GHSW)

The Geodesic Hyperbolic Sliced-Wasserstein (GHSW) distance utilizes the orthogonal projection of points onto a geodesic. For a point $x$ and a geodesic $\gamma$, the projection is the point on $\gamma$ that minimizes the hyperbolic distance to $x$.

Horospherical Projection (HHSW)

The Horospherical Hyperbolic Sliced-Wasserstein (HHSW) distance projects points along horospheres. Horospheres can be visualized as the limit of spheres whose centers move to infinity. This projection is mathematically linked to the Busemann function, which characterizes directions in hyperbolic space.

Comparison of projection methods in the Poincaré ball model: (a) Euclidean, (b) Geodesic, and (c) Horospherical.

By projecting data onto a 1D geodesic, the complex high-dimensional Wasserstein problem is reduced to a simple 1D problem with a closed-form solution based on sorting. The final distance is computed by averaging these 1D distances over multiple random geodesic directions.

Visualization of the projection mechanism on the Lorentz (hyperboloid) model.

Empirical Evaluation

The proposed methods were evaluated across several benchmarks, demonstrating both efficiency and performance gains.

Computational Efficiency

Both GHSW and HHSW maintain the $O(n \log n)$ complexity of the 1D Wasserstein distance, making them significantly faster than exact OT solvers. This scalability allows for the comparison of large-scale hyperbolic datasets that were previously computationally prohibitive.

Deep Hyperbolic Classification

In image classification tasks using CIFAR-10 and CIFAR-100, the authors employed a "Deep Classification with Prototypes" approach. Hyperbolic SW variants consistently outperformed existing baselines. For instance, on CIFAR-100 with 10-dimensional embeddings, HHSW achieved an accuracy of 62.80%, surpassing the PeBuse baseline of 59.19%.

Gradient Flows and Generative Modeling

The study of gradient flows revealed that hyperbolic-aware slicing leads to more stable and geometrically intuitive trajectories compared to Euclidean slicing applied naively to hyperbolic coordinates. This was further demonstrated in the training of Hyperbolic Sliced-Wasserstein Autoencoders (HHSWAE) on MNIST, where the latent space effectively captured the hierarchical nature of the data.

Conclusion

This work introduces a principled framework for Sliced-Wasserstein distances in hyperbolic space. By leveraging geodesic and horospherical projections, GHSW and HHSW provide computationally efficient and geometrically sound discrepancies for hierarchical data. These tools open new avenues for applying optimal transport in hyperbolic deep learning, from generative modeling to large-scale classification.

Summary updated

Abstract

Hyperbolic space embeddings have been shown beneficial for many learning tasks where data have an underlying hierarchical structure. Consequently, many machine learning tools were extended to such spaces, but only few discrepancies to compare probability distributions defined over those spaces exist. Among the possible candidates, optimal transport distances are well defined on such Riemannian manifolds and enjoy strong theoretical properties, but suffer from high computational cost. On Euclidean spaces, sliced-Wasserstein distances, which leverage a closed-form solution of the Wasserstein distance in one dimension, are more computationally efficient, but are not readily available on hyperbolic spaces. In this work, we propose to derive novel hyperbolic sliced-Wasserstein discrepancies. These constructions use projections on the underlying geodesics either along horospheres or geodesics. We study and compare them on different tasks where hyperbolic representations are relevant, such as sampling or image classification.
BibTeX
@inproceedings{bonet2023hyperbolic,
  author = {Clément Bonet and Laetitia Chapel and Lucas Drumetz and Nicolas Courty},
  title = {{Hyperbolic Sliced-Wasserstein via Geodesic and Horospherical Projections}},
  booktitle = {TAG-ML},
  year = {2023},
  url = {https://openreview.net/forum?id=g4XVO7gGOk},
}
Figure