Expanding the Hyperbolic Kernels: A Curvature-aware Isometric Embedding View

Meimei Yang, Pengfei Fang, Hui XueIJCAI 2023 · DOI

Introduction

Representing hierarchical or tree-like data—common in social networks and biological taxonomies—poses a significant challenge for standard Euclidean geometry. Because the volume of hyperbolic space grows exponentially with its radius, it provides a much more natural fit for such structures. However, applying powerful kernel methods to hyperbolic space has historically been difficult, often resulting in high distortion or inflexible models with fixed curvature.

Methodology: A Curvature-aware Isometric Bridge

The researchers address these limitations by proposing a novel curvature-aware isometric embedding. This approach creates a distortion-free bridge between the Poincaré ball (a model of hyperbolic space) and a specialized Reproducing Kernel Hilbert Space (RKHS) known as the Drury-Arveson Hardy space.

The isometric mapping from the Poincaré ball to the Hardy space ensures distance preservation while enabling kernel methods.

By generalizing this mapping to include a variable curvature parameter ($c$), the model can adapt to the specific "tree-likeness" of a dataset. This framework allows for the derivation of a suite of new kernels, including curvature-aware versions of Linear (CHL), Polynomial (CHPoly), RBF (CHRBF), and Laplacian (CHLap) kernels, as well as an indefinite Sigmoid (CHSig) kernel.

Key Insights: The Role of Curvature

A central finding of this work is that "one size does not fit all" when it comes to curvature. The optimal curvature value varies significantly across different datasets. By treating curvature as a tunable parameter, the proposed kernels can be optimized for specific data geometries.

Performance of various hyperbolic kernels across different curvature values, demonstrating the importance of curvature awareness.

Furthermore, the study highlights the effectiveness of the indefinite CHSig kernel. While indefinite kernels are less common in standard machine learning, this kernel proved exceptionally robust, particularly in Zero-shot Learning (ZSL) tasks.

Experimental Results

The authors evaluated their approach across two primary domains:

Graph Learning

On five real-world node classification datasets (Facebook, Terrorist, Wiki, AC, and Cora ML), the hyperbolic kernels consistently outperformed their Euclidean counterparts. For example, the CHL kernel achieved a 20.3% improvement in mean accuracy over the standard Euclidean Linear kernel.

Zero-shot Learning (ZSL)

In ZSL benchmarks like CUB, AWA1, and AWA2, the proposed kernels—especially CHSig—set new performance standards. This demonstrates that hyperbolic geometry is highly effective for cross-modality matching, where visual features must be mapped to semantic attributes.

Conclusion and Future Directions

This research provides a mathematically rigorous yet flexible framework for hyperbolic kernel methods. By ensuring isometry and incorporating curvature awareness, the authors have bridged a critical gap in non-Euclidean machine learning. Future work may focus on automating curvature optimization and scaling these methods to massive datasets using approximation techniques like Nyström methods.

Summary updated

BibTeX
@inproceedings{yang2023expanding,
  author = {Meimei Yang and Pengfei Fang and Hui Xue},
  title = {{Expanding the Hyperbolic Kernels: A Curvature-aware Isometric Embedding View}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/497},
}
Figure