From Knowledge Graph Embedding to Ontology Embedding: Region Based Representations of Relational Structures

Víctor Gutiérrez-Basulto, Steven Schockaert

TL;DR

This paper bridges the gap between knowledge graph embeddings (KGEs) and ontological reasoning by proposing a novel framework where relations are represented as geometric regions. They demonstrate that popular KGE methods struggle to capture even simple logical rules, but show that convex regions can effectively represent a significant class of ontological rules (quasi-chained existential rules), leading to logically consistent and deductively closed knowledge representations.

What Problem Does This Paper Address?

Knowledge graphs (KGs) are powerful for representing factual information, and KGEs have become a popular method for completing them by learning low-dimensional vector representations. However, KGs often come with associated ontologies that define logical dependencies between relations (e.g., if A implies B, then B must also hold). Existing KGE methods, while good at finding plausible facts, often fail to incorporate or even model these ontological rules. This limitation hinders their ability to ensure logical consistency and perform deductive reasoning. The paper aims to understand the compatibility of KGEs with ontological rules and proposes a more principled approach.

Methods

The core idea is to view relations not just as vectors or matrices, but as geometric regions in a vector space. A triple (subject, predicate, object) is considered valid if the concatenated entity embeddings fall within the region defined for that predicate.

  • Region-Based View: Relations are mapped to regions in a vector space. A triple (e, R, f) is valid if the combined embedding of e and f falls within the region R.
  • Convex Regions: The paper focuses on using convex regions to represent relations. This is motivated by the fact that many existing KGE models implicitly learn convex representations.
  • Quasi-Chained Existential Rules: A specific class of ontological rules is identified as being representable by convex regions. These rules have a structured body where variables are introduced in a controlled manner.
  • Geometric Model: A knowledge base is considered to have a geometric model if its embeddings satisfy the rules of the ontology and are logically consistent.

Datasets

Not discussed in the paper. The paper focuses on theoretical contributions and framework development rather than empirical evaluation on specific datasets.

Key Insights and Ideas

  • KGEs Struggle with Rules: Many popular KGE models (like TransE and DistMult) are fundamentally incapable of modeling even simple logical rules, meaning they cannot learn or enforce these dependencies. This is a significant limitation for tasks requiring logical reasoning.
  • Regions as a Unifying Concept: Representing relations as geometric regions provides a powerful abstraction that can unify different KGE approaches and, crucially, connect them to ontological reasoning.
  • Convexity is Key (for a Subset of Rules): While arbitrary regions could represent anything, restricting to convex regions is both practical (aligns with existing KGEs) and sufficient for a meaningful class of ontological rules (quasi-chained existential rules).
  • Ontology-Aware Embeddings: The proposed framework allows for the direct incorporation of ontological knowledge into the embedding process, leading to representations that are not only predictive but also logically sound.

Key Results

  • Incompatibility of Popular Models: The paper formally demonstrates that translation-based and certain bilinear KGE models are not expressive enough to capture even basic logical rules.
  • Expressivity of Convex Regions: Convex geometric models can precisely represent ontologies expressed using quasi-chained existential rules. This means that any set of facts induced by such an embedding will be logically consistent and deductively closed with respect to the ontology.
  • Theoretical Foundation: The work provides a theoretical framework for understanding the expressivity of KGEs in relation to ontological rules, paving the way for more robust and logically sound embedding methods.

Limitations and Future Work

  • Beyond Quasi-Chained Rules: While convex regions work for quasi-chained rules, more complex existential rules require extensions, such as "extended geometric models" involving non-linear mappings or neural networks. The expressivity of these extended models is still an open research question.
  • Practical Implementation: The paper lays the theoretical groundwork. Developing practical, efficient, and scalable embedding models that leverage convex regions and handle arbitrary rules remains a significant challenge.
  • Dimensionality: For datalog programs, the required dimensionality of the convex geometric model can be proportional to the number of constants, which might be impractical. Finding lower-dimensional representations is an area for future work.
  • Integration with Neural Networks: Exploring how neural networks can be used to learn these geometric representations and checking their compliance with ontologies is a promising direction.

Summary updated

Abstract

Recent years have witnessed the successful application of low-dimensional vector space representations of knowledge graphs to predict missing facts or find erroneous ones. However, it is not yet well-understood to what extent ontological knowledge, e.g. given as a set of (existential) rules, can be embedded in a principled way. To address this shortcoming, in this paper we introduce a general framework based on a view of relations as regions, which allows us to study the compatibility between ontological knowledge and different types of vector space embeddings. Our technical contribution is two-fold. First, we show that some of the most popular existing embedding methods are not capable of modelling even very simple types of rules, which in particular also means that they are not able to learn the type of dependencies captured by such rules. Second, we study a model in which relations are modelled as convex regions. We show particular that ontologies which are expressed using so-called quasi-chained existential rules can be exactly represented using convex regions, such that any set of facts which is induced using that vector space embedding is logically consistent and deductively closed with respect to the input ontology.
BibTeX
@inproceedings{gutierrezbasulto2018from,
  author = {Víctor Gutiérrez-Basulto and Steven Schockaert},
  title = {{From Knowledge Graph Embedding to Ontology Embedding: Region Based Representations of Relational Structures}},
  booktitle = {KR},
  year = {2018},
  url = {https://openreview.net/forum?id=Bq8Lex2VNh},
}
Figure