Abstract
Hierarchical data representation is a cornerstone of many machine learning tasks, yet standard Euclidean embeddings often suffer from the "crowding effect" as tree depth increases. While hyperbolic geometries offer a natural solution, they are mathematically complex and difficult to generalize to logical ontologies. We present RegD, a framework that leverages depth-based dissimilarity metrics to enable Euclidean regions (boxes and balls) to emulate hyperbolic behavior. RegD achieves state-of-the-art results on taxonomies and biomedical ontologies, demonstrating that hyperbolic properties can be achieved through metric design rather than manifold constraints.
Introduction
Hierarchical data—such as "is-a" relationships in biological taxonomies or product categories in e-commerce—is notoriously difficult to represent in standard flat (Euclidean) vector spaces. As one moves down a tree, the number of nodes grows exponentially, leading to a "crowding effect" where Euclidean space lacks the capacity to keep child nodes distinct from their parents and siblings.
To address this, researchers often turn to hyperbolic geometry, which naturally expands as one moves away from the center, providing "infinite room" for hierarchies. However, hyperbolic methods are mathematically cumbersome and often restricted to specific geometric shapes like cones. This limitation makes them difficult to generalize to complex ontologies that contain logical relationships beyond simple hierarchies (e.g., "A and B together imply C"). RegD asks: Can we obtain the expressive power of hyperbolic space while remaining within the flexible, familiar confines of Euclidean regions?
The RegD Framework
RegD introduces a novel approach where hyperbolic behavior is treated as a metric property rather than a manifold property. By carefully designing distance functions that scale with the "depth" or size of an object, Euclidean space can be made as expressive as a curved manifold.
The framework utilizes two primary dissimilarity metrics:
- Depth Dissimilarity ($d_{dep}$): This core innovation measures similarity by accounting for the regions' size (volume or diameter). The authors formally prove that as regions shrink—representing deeper levels of a hierarchy—this metric behaves like hyperbolic distance, "stretching" the space near boundaries to emulate exponential growth.
- Boundary Dissimilarity ($d_{bd}$): A non-symmetric measure designed to capture set-inclusion. It quantifies the "cost" of moving one region into or out of another, making it ideal for representing parent-child relationships.
By combining these into a single energy function, RegD represents nodes as arbitrary Euclidean regions—such as boxes or balls—while maintaining the mathematical benefits of a hyperbolic manifold.
Evaluation and Performance
The authors evaluated RegD across two distinct domains: Directed Acyclic Graphs (DAGs) and complex Biomedical Ontologies.
Directed Acyclic Graphs (DAGs)
Using real-world taxonomies like WordNet Mammal and the Microsoft Concept Graph, RegD (specifically the box variant) consistently outperformed previous state-of-the-art hyperbolic methods. For instance, on the Mammal dataset in low dimensions ($d=2$), RegD achieved an F1 score of 64.9%, significantly higher than the 57.7% achieved by ShadowCones.
Biomedical Ontologies
To test generalization, the model was applied to large-scale ontologies such as GALEN, Gene Ontology (GO), and Anatomy (Uberon). These require handling logical conjunctions and existential quantifiers. RegD provided a dramatic boost to existing Euclidean baselines. On the Anatomy ontology, integrating RegD into the ELBE model increased the F1 score from 43.1% to 58.7%, a relative improvement of over 36%.
Key Insights
- Metric-Driven Expressiveness: Hyperbolic results do not strictly require calculus on a Poincaré disk; they can be achieved through distance functions that account for object depth.
- Plug-and-Play Generalization: Because RegD operates in Euclidean space, it is easily integrated into existing models. The authors demonstrated this by simply swapping distance functions in established ontology models.
- Efficiency in Low Dimensions: RegD remains effective in extremely low-dimensional spaces where standard Euclidean region methods typically fail due to crowding.
Conclusion and Future Work
RegD demonstrates that Euclidean regions can capture the complex structures of hierarchies and ontologies with the efficiency of hyperbolic space. While performance on highly complex ontologies like GALEN remains a challenge, the framework opens new avenues for integrating hierarchical embeddings with Large Language Models (LLMs) and expanding to a wider variety of region types. Future research will focus on more sophisticated logical modeling and the integration of RegD with Hyperbolic Neural Networks (HNNs).
Summary updated