The APVA-TURBO Approach To Question Answering in Knowledge Base

Yue Wang, Richong Zhang, Cheng Xu, Yongyi MaoCOLING 2018 · OpenReview

Abstract

Question Answering over Knowledge Bases (QA-KB) often struggles with the vast search space of potential relation paths. This paper introduces the APVA-TURBO framework, which enhances the standard Align-Predict-Answer (APA) architecture with a dedicated verification mechanism (V) and an iterative "turbo training" procedure. By jointly optimizing path generation and verification, the approach achieves a significant 7.7% F1 score improvement on the WebQuestions benchmark, setting a new state-of-the-art.

Introduction

Knowledge Base Question Answering (QA-KB) aims to retrieve precise answers from structured databases like Freebase in response to natural language queries. A common paradigm is the "APA" architecture: aligning the subject entity, predicting the relation path, and retrieving the answer. However, path prediction remains a significant bottleneck due to the sparsity of training data relative to the massive space of possible relation sequences.

The APVA-TURBO approach addresses this by introducing a verification component and a novel iterative training loop. This dual strategy allows the model to not only generate candidate paths but also rigorously verify them against the original question, leading to substantial performance gains.

Methodology

The APVA Architecture

The APVA framework extends the standard pipeline with a verification step. It consists of four primary modules:

  1. Entity Alignment: Identifies the subject entity within the question.
  2. Path Prediction (ENC-DEC): A Gated Recurrent Unit (GRU) encoder-decoder generates candidate relation paths.
  3. Subject-Path Verification (CLS): A binary classifier (Multilayer Perceptron) that evaluates the likelihood that a specific (question, path) pair is correct.
  4. Object Answering: An SVM-based classifier that retrieves the final answer entities once a path is validated.
The probabilistic framework of APVA, illustrating the dependencies between the question q, subject s, path r, and verification variable y.

Turbo Training

To optimize these components, the authors propose "Turbo Training," an iterative procedure based on coordinate descent. Unlike static training, this method allows the encoder, decoder, and classifier to co-evolve.

The iterative "Turbo Training" loop for joint optimization of the encoder, decoder, and verification classifier.

The process begins with a base phase to initialize the path predictor, followed by a loop:

  • Train CLS: The encoder is fixed while the verification classifier is trained.
  • Train ENC: The classifier and decoder are fixed while the encoder is updated to better support the verification task.
  • Train DEC: The decoder is updated to refine path generation based on the improved representations.

Evaluation and Results

Experimental Setup

The model was evaluated using the WebQuestions (WQ) and SimpleQuestions (SQ) datasets, both mapped to the Freebase (FB2M) knowledge base. WebQuestions is particularly challenging as it often involves multi-hop relations and multiple answers.

Performance

APVA-TURBO demonstrated superior performance across benchmarks:

  • WebQuestions: Achieved an F1 score of 63.4%, a 7.7% improvement over previous state-of-the-art models.
  • SimpleQuestions: Reached 81.5% accuracy, outperforming standard APA models.

Ablation studies confirmed the necessity of both the verification component and the iterative training. Removing the "Turbo" loop (APVA-BASE) reduced the WQ F1 score to 59.7%, while removing verification entirely (APA-BASE) dropped it to 53.8%.

Discussion

The success of APVA-TURBO highlights that verification is a critical missing piece in generative QA-KB models. By adding a discriminative "sanity check," the system becomes more robust to the noise inherent in large-scale knowledge bases. The iterative training ensures that the internal representations (the encoder) are specifically tuned to distinguish between factually correct paths and spurious ones.

Limitations

The model currently assumes a uniform relation path for all answers to a single question. While effective for current benchmarks, this may limit performance on complex queries where answers are derived through heterogeneous logic paths. Future work could explore extending the framework to handle diverse answer paths and more complex iterative optimization strategies.

Summary updated

BibTeX
@inproceedings{wang2018apva,
  author = {Yue Wang and Richong Zhang and Cheng Xu and Yongyi Mao},
  title = {{The APVA-TURBO Approach To Question Answering in Knowledge Base}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
  url = {https://openreview.net/forum?id=pY5AvdI64},
}
Figure