TL;DR
Researchers at BenevolentAI have introduced PseudoE, a framework that extends pseudo-Riemannian manifold embeddings to complex, multi-relational knowledge graphs. By treating relations as transformations or "spacetime slices" within a multi-time manifold, the model captures non-metric graph structures more effectively than traditional Euclidean or Hyperbolic models, showing particular promise in biological drug discovery.
What Problem Does This Paper Address?
Knowledge Graphs (KGs) are the backbone of modern AI applications, from search engines to drug discovery. They represent entities (nodes) and their various relationships (edges). Traditionally, we embed these graphs into Euclidean space. However, Euclidean space is "metric"—it obeys strict rules like the triangle inequality.
Real-world data often breaks these rules. For example, in a hierarchy, a "grandfather" relation isn't just the sum of two "father" relations in a simple geometric sense. While Hyperbolic geometry has recently become popular for modeling hierarchies, it is still a Riemannian (metric) space.
Previous work introduced pseudo-Riemannian embeddings (like Minkowski space used in physics), which use an indefinite metric. This allows for "time" dimensions and "lightcones," making them perfect for directed, non-metric relationships. The gap? These models were previously limited to graphs with only one type of relation. This paper tackles the challenge of scaling these "spacetime" embeddings to multi-relational KGs.
Methods
The authors propose PseudoE, which incorporates multiple relations using two primary geometric strategies:
- Relations as Endomorphisms: Similar to standard models like TransE or DistMult, relations are modeled as transformations (translations or scaling) that move node embeddings within the manifold.
- Relations as Spacetime Submanifolds: This is the more novel approach. The authors define a manifold with multiple time dimensions. Each specific relation "projects" the data onto a specific 1-time submanifold. Essentially, different relations "see" different versions of spacetime.
- Wick-Rotation Interpolation: To handle relations that do behave like standard metrics (e.g., "is-similar-to"), they introduce a mixing parameter ($\beta$). This allows the model to smoothly interpolate between a pseudo-Riemannian "lightcone" structure and a standard Riemannian (Euclidean) isotropic structure.
- Learned Biases: They add node-specific and relation-specific bias terms. In this geometry, these act like "cones of influence," defining how far a node's reach extends in the graph.
Datasets
The model was evaluated on three benchmark datasets:
- FB15K-237: A standard knowledge graph subset of Freebase containing general world facts.
- WN18RR: A subset of WordNet focusing on lexical relations between English words.
- Hetionet (small): A specialized biological knowledge graph linking compounds, diseases, genes, and symptoms. This is particularly relevant for the authors' focus on drug discovery.
Key Insights and Ideas
- The Geometry of Directionality: The "lightcone" structure of pseudo-Riemannian space provides a natural inductive bias for directed edges. If node B is in the "future" lightcone of node A, the relationship is probable; if it's in the "elsewhere" region, it's not.
- Flexibility via Wick-Rotation: Not all relations are created equal. By using Wick-rotation, the model can adapt. For directed causal relations (like "drug causes side effect"), it uses the sharp boundaries of the lightcone. For symmetric relations (like "gene is associated with disease"), it can soften those boundaries to look more like standard Euclidean distance.
- Factoring out Popularity: The authors found that node biases correlate strongly with node degree (how "popular" a node is). By adjusting these biases at test time, researchers can "de-bias" the model to find novel, understudied connections—a "holy grail" in scientific discovery.
Key Results
- Competitive Performance: PseudoE is highly competitive, often ranking 1st or 2nd against state-of-the-art linear and flat-space models across all three datasets.
- Biological Superiority: On the Hetionet dataset, PseudoE (specifically the version combining both strategies) outperformed all baselines in link prediction (MRR of 0.544 vs. 0.527 for the next best).
- Qualitative Discovery: In a case study on breast cancer, the authors showed that by "turning down" the node bias, the model shifted from predicting well-known, high-degree genes (like TNF or TP53) to identifying understudied but biologically plausible targets (like DTX3).
Limitations and Future Work
- Flat Manifolds: This study restricted itself to "trivial flat" pseudo-Riemannian manifolds (Minkowski space). The authors acknowledge that extending this to curved pseudo-Riemannian spaces (like Anti-de Sitter or de Sitter spaces) is a logical next step that could capture even more complex hierarchies.
- Projection Complexity: The current "spacetime submanifold" approach uses simple linear combinations of time coordinates. More expressive, non-linear projections could potentially improve performance on datasets with a massive number of distinct relation types.
- Task Variety: The evaluation focused primarily on link prediction. Future work could explore how these embeddings perform in node classification or graph clustering tasks.
Summary updated