Entity Prediction in Knowledge Graphs with Joint Embeddings

Matthias Baumgartner, Daniele Dell’Aglio, Abraham BernsteinNAACL 2021

TL;DR

This paper tackles the challenge of automatically enriching Knowledge Graphs (KGs) by identifying and integrating new entities that are described in text but not explicitly present in the KG. They propose a novel method using joint embedding models to represent both KGs and text in a shared space, enabling the prediction of new entities and their relationships, which is then refined with user feedback and graph features.

What Problem Does This Paper Address?

Knowledge Graphs (KGs) are powerful tools for representing structured information, but they are often incomplete and require continuous updates as the real world evolves. A key challenge is "entity prediction": discovering and integrating new entities into an existing KG when they are described in external text sources (like Wikipedia articles) but not explicitly mentioned in the KG itself. Traditional KG completion methods focus on predicting missing links between existing entities, but they cannot identify entirely new entities. Existing approaches for extracting information from text often rely on explicit mentions of entity names, which is not always the case. This paper aims to bridge this gap by proposing a method to infer the existence of new entities and their relationships from textual descriptions, even when the entities are not explicitly named.

Methods

The core of the proposed solution lies in joint embedding models. These models learn to represent both KG entities/relations and text documents in a shared, low-dimensional vector space. This shared space allows for comparing the similarity between documents and KG elements. Specifically, the authors leverage models like KADE and StarSpace, which learn embeddings for KG triples (head, relation, tail) and document text.

The entity prediction process is broken down into two main subproblems:

  1. Novelty Detection: Identifying "surplus" documents (those not directly mapped to an existing KG entity) that describe genuinely new entities versus those that might describe an existing entity with an unknown relationship. This is framed as a binary classification problem, using features derived from the document embeddings and their distances to other document embeddings.
  2. Triple Reconstruction: Once a new entity (represented by its document embedding) is identified, the method aims to predict new triples (entity, relation, tail) involving this new entity. This is achieved by:
  3. Triple Loss: Using the joint embedding space to rank potential triples based on their plausibility (e.g., how well the new entity's embedding aligns with a potential relation and tail entity's embedding).
  4. User Feedback (UF): Incorporating human input to refine relation prediction. The system presents a set of candidate relations, and the user selects correct ones, which helps train a classifier to identify likely relations for the new entity.
  5. Graph Features: Enhancing the process with structural information from the KG. This includes using Association Rule Learning (CO) to improve relation selection and BM25 weighting to better identify relevant tail entities for a given relation.

Datasets

The authors evaluate their methods on two popular KG datasets:

  • FB15k-237: A subset of Freebase.
  • DBP50: A subset of DBpedia.

For these datasets, they create corresponding document corpora by selecting random sections from Wikipedia articles. To ensure the methods learn from implicit descriptions, they replace explicit mentions of entity names with the generic term "entity." They also randomly sample entities and their corresponding documents to create a set of "surplus" documents for novelty detection and triple reconstruction.

Key Insights and Ideas

  • Joint Embeddings as a Bridge: The fundamental insight is that joint embedding models can effectively bridge the gap between unstructured text and structured KGs, enabling the inference of new entities and relationships that are not explicitly stated.
  • Implicit Descriptions are Key: The paper highlights the importance of handling entities described implicitly in text, moving beyond traditional methods that rely on explicit naming.
  • User Feedback is Crucial for Refinement: While joint embeddings provide a strong foundation, user feedback significantly improves the precision and recall of triple reconstruction, especially in identifying correct relations.
  • Graph Structure Complements Embeddings: Integrating graph features (like association rules and BM25) further enhances the prediction accuracy by leveraging KG structure, addressing different aspects of the triple reconstruction problem.
  • KADE Outperforms StarSpace: The KADE model, with its more constrained document modeling, shows superior performance in novelty detection compared to StarSpace.

Key Results

  • Novelty Detection: KADE embeddings achieve substantially higher accuracy (e.g., 0.640 on FB15k-237) in distinguishing novel from unassociated documents compared to a bag-of-words model (e.g., 0.505 on FB15k-237). KADE also significantly reduces type-I errors, preventing redundant additions to the KG.
  • Triple Reconstruction (TopN): Joint embedding models (KADE and StarSpace) perform significantly better than random baselines in predicting triples, though still below an "Oracle" baseline that uses ground truth. KADE generally outperforms StarSpace.
  • Impact of User Feedback: User feedback (UF and UF-s) substantially improves precision and recall. UF-s, which guarantees an initial relation, shows an average increase of 13.43% in precision and 9.83% in recall over UF.
  • Impact of Graph Features: Combining user feedback with graph features (BM+CO) leads to significant improvements. On FB15k-237, BM+CO improves recall by 23.11% and precision by 22.02% over methods without graph features. The CO feature primarily boosts recall, while BM boosts precision.
  • User Workload: The proposed methods, particularly UF-s, involve a manageable number of user judgments, often significantly less than a theoretical upper bound.

Limitations and Future Work

  • Scalability of Incremental Updates: The current method restores entities and triples independently. The authors suggest future work on developing an approach that incrementally updates the KG and its embeddings.
  • Integration of Explicit Information: The method relies on structural patterns and implicit mentions. Future work could explore integrating explicit information like schema or semantics of relations and entities.
  • Extension to Other Modalities: The approach could be extended to incorporate other data sources, such as images, for entity prediction.
  • Predicting Novel Relations: While the focus is on entity prediction, the authors suggest extending the work to predict novel relations as well.
  • Dataset Sparsity: On sparser datasets like DBP50, the impact of graph features can be less pronounced or more variable due to limited samples for accurate frequency estimation.

Summary updated

BibTeX
@inproceedings{baumgartner2021entity,
  author = {Matthias Baumgartner and Daniele Dell’Aglio and Abraham Bernstein},
  title = {{Entity Prediction in Knowledge Graphs with Joint Embeddings}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure