Integrating Ontologies and Vector Space Embeddings Using Conceptual Spaces (Invited Paper)

Zied Bouraoui, Víctor Gutiérrez-Basulto, Steven SchockaertAIB 2022 · OpenReview

TL;DR

This paper explores how to bridge the gap between symbolic knowledge (ontologies) and vector space embeddings, two powerful but distinct AI paradigms. It proposes using "conceptual spaces" as an intermediate layer to integrate their strengths, enabling more flexible and interpretable AI systems that can reason with both logical rules and learned similarities.

What Problem Does This Paper Address?

Artificial Intelligence often relies on two main ways to represent knowledge: ontologies, which provide precise logical relationships between concepts, and vector space embeddings, which capture similarities and analogies between entities. While ontologies excel at logical reasoning and interpretability, they struggle with incomplete knowledge and nuanced relationships. Vector embeddings, on the other hand, are great at capturing similarity and analogy but lack the logical rigor and transparency of ontologies. This paper addresses the challenge of combining these complementary strengths to create more robust and intelligent AI systems. The core problem is how to integrate these fundamentally different representation methods to leverage the best of both worlds.

Methods

The paper presents an overview of recent work that uses the theory of conceptual spaces as a unifying framework. Conceptual spaces, proposed by Gärdenfors, represent concepts as regions rather than points (like in vector embeddings) and use "quality dimensions" to capture semantic features.

The authors discuss several approaches:

  • Learning Conceptual Space Representations: Strategies for learning these region-based representations from data are explored, focusing on how to identify meaningful "quality dimensions" and organize them into "domains." This includes methods for learning Gaussian representations of concepts and exploiting "contrast sets" and "conceptual neighborhood" to define concept regions.
  • Geometric Models of Relational Rules: The paper extends conceptual spaces to model relational knowledge (rules involving multiple entities). This is done by representing k-ary relations as convex regions in the product of k conceptual spaces. This allows for a geometric interpretation of logical rules, aiming for a tight integration of symbolic and geometric representations.
  • Looser Integration via Similarity: Recognizing the limitations of tight integration, the paper also discusses methods where vector space embeddings are used to derive similarity measures. These similarities then support flexible reasoning with ontologies, enabling inductive inferences and plausible rule generation that goes beyond strict logical deduction. This includes approaches like interpolative and analogical reasoning.

Datasets

Not discussed in the paper. The paper focuses on theoretical frameworks and methods, referencing existing work that likely used various datasets for concept and relation learning (e.g., BabelNet concepts, movie embeddings), but specific datasets used in the presented work are not detailed.

Key Insights and Ideas

  • Conceptual Spaces as a Bridge: The core idea is that conceptual spaces, with their region-based representations and semantic dimensions, can act as a natural interface between symbolic ontologies and vector space embeddings.
  • Beyond Vectors for Concepts: Representing concepts as regions (rather than just points/vectors) in a conceptual space allows for a more nuanced capture of concept boundaries and relationships, aligning better with human cognition.
  • Interpretable Dimensions from Embeddings: Learned vector space embeddings can be "disentangled" to identify interpretable "quality dimensions," making the representations more understandable and useful for reasoning.
  • Geometric Interpretation of Rules: Logical rules can be translated into geometric constraints on regions in conceptual spaces, offering a way to tightly integrate symbolic logic with geometric representations.
  • Similarity for Plausible Reasoning: Vector embeddings can provide valuable similarity information that, when combined with ontologies, allows for inductive and analogical reasoning, enabling AI systems to infer plausible knowledge beyond strict deduction.

Key Results

The paper is an overview of recent work, so it doesn't present a single set of quantitative results. Instead, it highlights the potential of the discussed approaches:

  • Improved Interpretability: Identifying quality dimensions from embeddings leads to more interpretable AI models.
  • Flexible Reasoning: The proposed methods enable more flexible reasoning capabilities, allowing for inductive and analogical inferences that are difficult with purely symbolic or purely vector-based approaches.
  • Modeling Relational Knowledge: The geometric approach offers a way to model relational rules that are challenging for traditional conceptual spaces.
  • Bridging Paradigms: The work demonstrates concrete strategies for combining the precision of ontologies with the similarity-capturing power of vector embeddings.

Limitations and Future Work

  • Learning Region-Based Representations: A significant challenge is the practical difficulty of learning accurate region-based representations of concepts from data, especially with limited instances.
  • Modeling Relational Knowledge: While geometric models can represent relational rules, integrating them tightly with symbolic representations has limitations, particularly with complex rules or when requiring convex regions.
  • Negation and Disjunction: Modeling negation and disjunction within geometric frameworks remains an open challenge.
  • Computational Complexity: Reasoning with interpolation in conceptual spaces can be computationally more demanding than in pure description logics.
  • Future Directions: The authors suggest exploring larger fragments of existential rules that can be modeled with convex regions, relaxing the convexity assumption for learnability, and developing nonmonotonic extensions for analogical reasoning to handle potential conflicts. Further research is also needed on the computational complexity of these new reasoning paradigms.

Summary updated

Abstract

Ontologies and vector space embeddings are among the most popular frameworks for encoding conceptual knowledge. Ontologies excel at capturing the logical dependencies between concepts in a precise and clearly defined way. Vector space embeddings excel at modelling similarity and analogy. Given these complementary strengths, there is a clear need for frameworks that can combine the best of both worlds. In this paper, we present an overview of our recent work in this area. We first discuss the theory of conceptual spaces, which was proposed in the 1990s by Gärdenfors as an intermediate representation layer in between embeddings and symbolic knowledge bases. We particularly focus on a number of recent strategies for learning conceptual space representations from data. Next, building on the idea of conceptual spaces, we discuss approaches where relational knowledge is modelled in terms of geometric constraints. Such approaches aim at a tight integration of symbolic and geometric representations, which unfortunately comes with a number of limitations. For this reason, we finally also discuss methods in which similarity, and other forms of conceptual relatedness, are derived from vector space embeddings and subsequently used to support flexible forms of reasoning with ontologies, thus enabling a looser integration between embeddings and symbolic knowledge.
BibTeX
@inproceedings{bouraoui2022integrating,
  author = {Zied Bouraoui and Víctor Gutiérrez-Basulto and Steven Schockaert},
  title = {{Integrating Ontologies and Vector Space Embeddings Using Conceptual Spaces (Invited Paper)}},
  booktitle = {AIB},
  year = {2022},
  url = {https://openreview.net/forum?id=RwPrSbxly3},
}
Figure