Modeling Global Semantics for Question Answering over Knowledge Bases

Peiyun Wu, Yunjie Wu, Linjuan Wu, Xiaowang Zhang, Zhiyong FengIJCNN 2021 · OpenReview

TL;DR

This paper introduces gRGCN, a novel model for question answering over knowledge bases (KBQA) that captures "global semantics" by jointly considering the structure of questions and their corresponding query graphs. By leveraging Relational Graph Convolutional Networks (RGCN) and a hierarchical attention mechanism, gRGCN significantly outperforms existing methods on complex KBQA tasks.

What Problem Does This Paper Address?

Question Answering over Knowledge Bases (KBQA) aims to answer natural language questions by querying structured knowledge bases. A crucial step in many KBQA systems is semantic parsing, which transforms a question into a formal query, often represented as a "query graph." Existing semantic parsing approaches for KBQA tend to focus on matching individual relations between the question and the query graph, often overlooking the underlying structural relationships within the question itself. This limitation becomes particularly problematic for complex questions, where intricate structures can significantly influence the intended meaning. The paper identifies a gap in effectively modeling both the relational and structural semantics of questions and query graphs simultaneously, especially when dealing with multi-relational directed graphs.

Methods

The proposed model, gRGCN, tackles this challenge by integrating multiple components:

  • Question Representation Encoder: This module encodes the question's syntactic dependency tree and word sequence into a unified representation. It uses an RGCN to process the dependency tree and then combines these embeddings with word embeddings, employing an attention mechanism to weigh the importance of different words.
  • Structure Semantics Extractor: This module focuses on capturing the global structure of query graphs. It utilizes a relation-attention-based RGCN layer to learn embeddings for the query graph's structure, considering the relationships between entities and relations.
  • Relational Semantics Extractor: This component extracts semantic representations of relations within the query graphs. It employs a hierarchical attention mechanism that leverages WordNet to disambiguate word-level relations and capture finer-grained semantic nuances. This helps to reduce noise and improve the representation of relations.

The core innovation lies in the fusion of these extracted structure and relational semantics. gRGCN combines these representations to form a "global semantic" understanding, which is then used to match questions with candidate query graphs.

Datasets

The paper evaluates gRGCN on several popular KBQA datasets:

  • ComplexQuestion (CompQ): Designed for complex questions with diverse structures and expressions.
  • WebQSP-WD (WebQSPS): A corrected version of WebQSP, adapted for Wikidata.
  • QALD-7: A dataset specifically for testing KBQA models on complex and simple questions over Wikidata.
  • Simple Question (SimpQ): Used for complementary experiments to assess the model's robustness on simpler questions.

The experiments primarily use Wikidata and FB2M as knowledge bases.

Key Insights and Ideas

  • Global Semantics is Key: The paper strongly advocates for the importance of "global semantics," which encompasses both the structural relationships within a question and the relational semantics of query graphs. Simply matching individual relations is insufficient for complex questions.
  • RGCN for Structure: Relational Graph Convolutional Networks (RGCN) are effectively employed to capture the structural semantics of both questions (via dependency trees) and query graphs.
  • Hierarchical Attention for Relations: A hierarchical attention mechanism, enhanced by WordNet, proves crucial for disambiguating and enriching the representation of relations, mitigating noise from word-level semantics.
  • Fusion is Powerful: The joint fusion of structural and relational semantics leads to a more comprehensive understanding, significantly boosting performance.

Key Results

gRGCN demonstrates superior performance across multiple datasets and metrics (precision, recall, F1-score).

  • On WebQSPS, gRGCN achieved a 12.4% higher F1-score compared to the GGNN baseline.
  • For QALD-7, the improvement was even more substantial, with a 41.9% higher F1-score over GGNN.
  • On CompQ, gRGCN showed a 18.2% higher F1-score compared to GGNN.
  • Ablation studies confirmed the effectiveness of each module: the question representation encoder, the graph representation, and the integration of relational types all contributed to performance gains.
  • Even on simple questions, gRGCN maintained competitive accuracy, demonstrating its robustness.

Limitations and Future Work

  • Semantic Ambiguity and Complexity: The paper acknowledges that for some highly complex or semantically ambiguous questions, generating a comprehensive graph space for analysis remains a challenge.
  • Entity Linking Errors: Errors in entity linking can lead to incorrect semantic graph generation, impacting the overall performance.
  • Dataset and KB Incompatibility: Issues related to the compatibility between datasets and knowledge bases (e.g., Wikidata vs. Freebase IDs) and potential defects in datasets were noted as sources of error.

The authors suggest future work could involve extending the model to handle even more complex practical questions and exploring further improvements in question representation and graph generation.

Summary updated

Abstract

Semantic parsing, as an important approach to question answering over knowledge bases (KBQA), transforms a question into the complete query graph for further generating the correct logical query. Existing semantic parsing approaches mainly focus on relations matching with paying less attention to the underlying internal structure of questions (e.g., the dependencies and relations between all entities in a question) to select the query graph. In this paper, we present a relational graph convolutional network (RGCN)-based model gRGCN for semantic parsing in KBQA. gRGCN extracts the global semantics of questions and their corresponding query graphs, including structure semantics via RGCN and relational semantics (label representation of relations between entities) via a hierarchical relation attention mechanism. Experiments evaluated on benchmarks show that our model outperforms off-the-shelf models.
BibTeX
@inproceedings{wu2021modeling,
  author = {Peiyun Wu and Yunjie Wu and Linjuan Wu and Xiaowang Zhang and Zhiyong Feng},
  title = {{Modeling Global Semantics for Question Answering over Knowledge Bases}},
  booktitle = {IJCNN},
  year = {2021},
  url = {https://openreview.net/forum?id=5XyFYI1MQxA},
}
Figure