Knowledge Base Relation Detection via Multi-View Matching

Yang Yu, Kazi Saidul Hasan, Mo Yu, Wei Zhang, Zhiguo WangADBIS (Short Papers and Workshops) 2018 · OpenReview

TL;DR

This paper tackles the challenge of identifying the correct relation in a knowledge base (KB) that corresponds to a natural language question. They propose a novel "multi-view matching" approach that leverages not only relation names but also entity types associated with those relations, achieving state-of-the-art results on benchmark datasets.

What Problem Does This Paper Address?

Knowledge Base Question Answering (KBQA) systems aim to answer user questions by querying structured knowledge bases (like Freebase or DBpedia). A crucial step in this process is "relation detection," which involves identifying which KB relation best matches the intent of a given question. This is a difficult task for several reasons: the sheer number of relations in a KB (often thousands), and the fact that relations in a test set might not have been seen during training (zero-shot learning). Existing methods often struggle with this scale and the need to generalize. This paper focuses specifically on improving the accuracy of this relation detection component.

Methods

The core innovation is a multi-view matching framework. Instead of relying solely on the relation's name, the model considers multiple "views" of information from both the question and the KB relation. These views include:

  • Entity Pair: Information about the entities mentioned in the question and their corresponding KB entities.
  • Abstracted Question and Relation Name: The question is "abstracted" by replacing entity mentions with a special token, allowing the model to focus on the relation name's context within the question.
  • Abstracted Question and Relation Tail Entity Types: This is a key novel contribution. The model explicitly uses the types of entities that a relation typically points to (e.g., if a relation is "located in," its tail entity type is likely "location"). This provides an additional signal for matching.

For each view, the model employs a bilateral multi-perspective matching (BiMPM) module. This module, adapted from text matching tasks, compares an anchor sequence (e.g., question) with a target sequence (e.g., relation name or entity type) using multiple matching strategies (full matching, max-pooling, attentive matching) and multiple "perspectives" (learned weightings of dimensions). The outputs from these BiMPM modules across all views are then aggregated to produce a final prediction. The entire system is trained end-to-end.

Datasets

The paper evaluates its approach on two standard KBQA datasets:

  • SimpleQuestions (SQ): Contains single-relation questions, where a head entity and a tail entity are connected by one relation.
  • WebQSP (WQ): Includes both single and multi-relation questions, where a path between a head and tail entity might involve multiple relations.

The authors use subsets of Freebase (FB2M for SQ) and the entire Freebase KB for WQ. They adopt standard train/validation/test splits and data creation strategies from prior work.

Key Insights and Ideas

  • Leveraging Entity Types is Crucial: The most significant insight is that incorporating the types of entities a relation typically connects to (tail entity types) dramatically improves relation detection accuracy. This provides a semantic signal that relation names alone might not convey.
  • Multi-View Matching Captures Diverse Signals: By considering multiple views (entity pairs, relation names, entity types), the model can capture a richer set of signals, leading to more robust matching.
  • Abstracting Questions Helps Focus: Replacing entity mentions in questions with a special token allows the model to better focus on the semantic content of the question and its relation to the candidate relation name.
  • Character Embeddings Offer Limited Benefit Here: The authors found that adding character embeddings on top of word embeddings did not significantly improve performance. They hypothesize this is because the vocabulary of relations and entity types in KBQA is much smaller and less varied than in general text, limiting the utility of character-level features for capturing morphological variations.

Key Results

The proposed multi-view matching model achieves state-of-the-art results on both datasets:

  • WebQSP (WQ): Achieves 85.95% accuracy (Row 11) and 85.41% (Row 12), outperforming the previous best system by 3.42 points.
  • SimpleQuestions (SQ): Achieves 93.69% accuracy (Row 11) and 93.75% (Row 12), beating the previous best by 0.45 points.

The results also show that using relation and tail entity types as separate inputs (Rows 8-10) consistently outperforms concatenating them into a single input (Rows 6-7).

Limitations and Future Work

  • Character Embeddings: As mentioned, character embeddings did not provide a significant boost, suggesting that for this specific task with limited vocabularies, they might not be as impactful as in other NLP tasks.
  • Dataset Difficulty: The paper notes that WQ is generally more difficult than SQ due to multi-relation questions, more realistic question phrasing, and more complex KB reasoning required.
  • Future Work: While not explicitly detailed, the success of multi-view matching and the inclusion of entity types suggest future work could explore even more diverse views of information from the question and KB, potentially incorporating more sophisticated ways to represent and match these views. The authors also mention that extracting entity information separately for final prediction was useful, hinting at potential integration with downstream tasks.

Summary updated

Abstract

Relation detection is a core component for Knowledge Base Question Answering (KBQA). In this paper, we propose a KB relation detection model via multi-view matching which utilizes more useful information extracted from question and KB. The matching inside each view is through multiple perspectives to compare two input texts thoroughly. All these components are designed in an end-to-end trainable neural network model. Experiments on SimpleQuestions and WebQSP yield state-of-the-art results.
BibTeX
@inproceedings{yu2018knowledge,
  author = {Yang Yu and Kazi Saidul Hasan and Mo Yu and Wei Zhang and Zhiguo Wang},
  title = {{Knowledge Base Relation Detection via Multi-View Matching}},
  booktitle = {ADBIS (Short Papers and Workshops)},
  year = {2018},
  url = {https://openreview.net/forum?id=-jqUWNByATS},
}
Figure