Knowledge Representation with Conceptual Spaces

Steven SchockaertSemDeep @ COLING 2018 · OpenReview

TL;DR

This paper explores how to represent knowledge in a more structured and interpretable way than standard "entity embeddings." It proposes using "conceptual spaces," which go beyond simple similarity to explicitly model properties and concepts, enabling richer queries and better performance in unsupervised learning.

What Problem Does This Paper Address?

Traditional "entity embeddings" represent items (like movies) as vectors in a low-dimensional space, where similar items are close together. While useful for tasks like recommendations, they implicitly encode knowledge, making it hard to leverage for complex queries or in unsupervised settings. For example, knowing a movie's genre is implicitly encoded doesn't easily allow us to ask "find movies similar to Jurassic Park but scarier." This paper addresses the limitation of implicit knowledge representation by proposing a more structured approach.

Methods

The paper introduces "conceptual spaces," inspired by Gärdenfors' work. Unlike standard entity embeddings, conceptual spaces feature:

  • Interpretable Dimensions: Dimensions within the space have clear meanings.
  • Grouped Facets: Dimensions are organized into meaningful groups (facets).
  • Explicit Properties and Concepts: Properties and concepts are represented as distinct, often "vague," regions within the space.

This structure allows for more direct querying based on specific features or concepts, and the paper discusses recent approaches for learning these representations, particularly within the FLEXILOG project, to make them more amenable to data-driven learning.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Beyond Similarity: Entity embeddings capture similarity well, but conceptual spaces aim to capture richer, structured knowledge about entities and their relationships.
  • Interpretable Knowledge: The explicit structure of conceptual spaces makes the learned representations more interpretable, allowing us to understand why certain entities are related.
  • Enabling Complex Queries: Conceptual spaces facilitate queries that go beyond simple similarity, such as finding entities with specific properties or combinations of features (e.g., "movies from the 90s with dinosaurs").
  • Unsupervised Learning Advantage: The structured nature of conceptual spaces makes them particularly valuable for unsupervised learning tasks, where implicit knowledge in standard embeddings is harder to exploit.

Key Results

The paper is a position/overview paper and does not present specific quantitative results. It outlines the potential of conceptual spaces for knowledge representation and mentions ongoing work within the FLEXILOG project to develop methods for learning them.

Limitations and Future Work

The paper acknowledges that learning conceptual spaces in a purely data-driven fashion is more challenging compared to standard entity embeddings. Future work, as indicated by the mention of the FLEXILOG project, focuses on developing more effective approaches for learning these structured representations from data.

Summary updated

Abstract

Entity embeddings are vector space representations of a given domain of interest. They are typically learned from text corpora (possibly in combination with any available structured knowledge), based on the intuition that similar entities should be represented by similar vectors. The usefulness of such entity embeddings largely stems from the fact that they implicitly encode a rich amount of knowledge about the considered domain, beyond mere similarity. In an embedding of movies, for instance, we may expect all movies from a given genre to be located in some low-dimensional manifold. This is particularly useful in supervised learning settings, where it may e.g. allow neural movie recommenders to base predictions on the genre of a movie, without that genre having to be specified explicitly for each movie, or without even the need to specify that the genre of a movie is a property that may have predictive value for the considered task. In unsupervised settings, however, such implicitly encoded knowledge cannot be leveraged. Conceptual spaces, as proposed by Grdenfors, are similar to entity embeddings, but provide more structure. In conceptual spaces, among others, dimensions are interpretable and grouped into facets, and properties and concepts are explicitly modelled as (vague) regions. Thanks to this additional structure, conceptual spaces can be used as a knowledge representation framework, which can also be effectively exploited in unsupervised settings. Given a conceptual space of movies, for instance, we are able to answer queries that ask about similarity w.r.t. a particular facet (e.g. movies which are cinematographically similar to Jurassic Park), that refer to a given feature (e.g. movies which are scarier than Jurassic Park but otherwise similar), or that refer to particular properties or concepts (e.g. thriller from the 1990s with a dinosaur theme). Compared to standard entity embeddings, however, conceptual spaces are more challenging to learn in a purely data-driven fashion. In this talk, I will give an overview of some approaches for learning such representations that have recently been developed within the context of the FLEXILOG project.
BibTeX
@inproceedings{schockaert2018knowledge,
  author = {Steven Schockaert},
  title = {{Knowledge Representation with Conceptual Spaces}},
  booktitle = {SemDeep @ COLING},
  year = {2018},
  url = {https://openreview.net/forum?id=yKhRtkoZeD},
}
Figure