Neural Generative Question Answering

Jun Yin, Xin Jiang, Zhengdong Lu, Lifeng Shang, Hang Li, Xiaoming Li

Abstract

This paper presents an end-to-end neural network model, named Neural Generative Question Answering (GENQA), that can generate answers to simple factoid questions, based on the facts in a knowledge-base. More specifically, the model is built on the encoder-decoder framework for sequence-to-sequence learning, while equipped with the ability to enquire the knowledge-base, and is trained on a corpus of question-answer pairs, with their associated triples in the knowledge-base. Empirical study shows the proposed model can effectively deal with the variations of questions and answers, and generate right and natural answers by referring to the facts in the knowledge-base. The experiment on question answering demonstrates that the proposed model can outperform an embedding-based QA model as well as a neural dialogue model trained on the same data.

Introduction: The Tension Between Fluency and Factuality

Traditional Question Answering (QA) systems typically struggle with a fundamental trade-off. Retrieval-based systems are precise because they pull raw facts directly from a database, but they often lack the linguistic flexibility to present those facts in natural, conversational language. Conversely, generation-based systems—such as standard Recurrent Neural Networks (RNNs)—are highly fluent but struggle to accurately "remember" discrete, isolated facts like specific names or dates. This often leads to "hallucinations" or poor performance on rare entities.

The authors of GENQA identify that distributed neural representations are excellent for capturing "smooth" patterns like syntax and style, but ill-suited for representing "discrete" concepts. GENQA aims to bridge this gap by creating a model that can "talk" like a human while "looking up" facts in a structured Knowledge Base (KB) in real-time.

The GENQA Architecture

The GENQA model is an end-to-end system integrated into an encoder-decoder framework. It consists of three primary components: the Interpreter, the Enquirer, and the Answerer.

The high-level architecture of the GENQA model.
  1. The Interpreter (Encoder): This component uses a bidirectional Gated Recurrent Unit (GRU) to transform the input question into a sequence of vector representations, capturing the semantic intent of the user.
  2. The Enquirer: Acting as a "lookup" mechanism, the Enquirer identifies candidate triples (subject, predicate, object) from the KB. It calculates a matching score between the question and these triples using either a Bilinear Model or a CNN-based Matching Model to determine which specific fact is most relevant to the query.
  3. The Answerer (Decoder): This is the core innovation of the model. It is a recurrent neural network that generates the final response by dynamically deciding where to source its information.

The Answerer: Bridging Language and Knowledge

The Answerer employs a hybrid decoder equipped with a latent "switch" variable ($z_t$). At each time step of the generation process, the model makes a probabilistic decision on whether to generate a common word or a specific entity from the knowledge base.

The hybrid decoder mechanism using a latent switch variable.

As shown in the decoder logic, the model can:

  • Generate a common word: Pull from a standard vocabulary for syntactic structure (e.g., "is", "born", "in").
  • Retrieve a KB-word: Extract the "object" from the chosen KB triple (e.g., "2.29m" or "Germany").

By separating the "language" component from the "knowledge" component, the model avoids the bottleneck of trying to store millions of unique entities within fixed-size neural weights. This joint training allows the Enquirer to learn which questions require specific facts based on the feedback from the Answerer.

Datasets and Empirical Study

To evaluate GENQA, the researchers developed a large-scale grounded dataset:

  • Knowledge Base: Mined from Chinese encyclopedia sites (Baidu Baike, Douban), containing approximately 11 million triples.
  • QA Pairs: Extracted 235 million pairs from community QA sites.
  • Grounded Dataset: A refined set of 720,000 instances where questions and answers are explicitly linked to KB triples.

The study compared GENQA against a pure neural dialogue model (NRM) and an embedding-based retrieval model. GENQA with a CNN-based enquirer achieved 52% accuracy, significantly outperforming the NRM (19%) and the embedding-based model (45%).

Key Insights and Results

  • CNNs for Fact Matching: Using a Convolutional Neural Network to match questions to triples proved superior to bilinear models, as CNNs better capture the salient features that point toward specific predicates.
  • Fluency and Naturalness: Qualitative analysis showed that GENQA successfully blends KB facts into natural sentences (e.g., "He plays for FC Barcelona" instead of just "FC Barcelona").
  • Generalization: The model demonstrated the ability to answer questions about facts not seen during training, indicating it learned the relationship between language and the KB structure rather than just memorizing specific pairs.

Limitations and Future Work

While successful for simple factoids, GENQA is currently limited to "forward relation" questions (Subject + Predicate = ?). It cannot yet perform complex reasoning across multiple triples. Additionally, in a small percentage of cases (8%), the model retrieved the correct fact but placed it within a linguistically incorrect context. Future research will focus on multi-turn dialogues and querying more complex, graph-structured knowledge bases.

Summary updated

Abstract

This paper presents an end-to-end neural network model, named Neural Generative Question Answering (GENQA), that can generate answers to simple factoid questions, based on the facts in a knowledge-base. More specifically, the model is built on the encoder-decoder framework for sequence-to-sequence learning, while equipped with the ability to enquire the knowledge-base, and is trained on a corpus of question-answer pairs, with their associated triples in the knowledge-base. Empirical study shows the proposed model can effectively deal with the variations of questions and answers, and generate right and natural answers by referring to the facts in the knowledge-base. The experiment on question answering demonstrates that the proposed model can outperform an embedding-based QA model as well as a neural dialogue model trained on the same data.
BibTeX
@inproceedings{yin2016neural,
  author = {Jun Yin and Xin Jiang and Zhengdong Lu and Lifeng Shang and Hang Li and Xiaoming Li},
  title = {{Neural Generative Question Answering}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
  url = {https://openreview.net/forum?id=BJbtZSMu-r},
}
Figure