Abstract
Modern reading comprehension systems excel at extracting short spans of text as answers, but these "factoids" often provide a robotic user experience in conversational interfaces. We propose a "plug-and-play" full-length answer generator that transforms these short spans into natural sentences using only the question and the extracted factoid. Utilizing a dual-encoder pointer-generator network and a new dataset of 315,000 triples, our system achieves a 74.05 BLEU score and an 86.25 ROUGE-L score, demonstrating significant improvements in conversational fluency without requiring access to the original source passage.
Introduction: Beyond Robotic Responses
Extractive Question Answering (QA) systems, typically trained on datasets like SQuAD, are highly effective at identifying specific entities or phrases within a document. However, responding to a user with an isolated phrase—such as answering "10th and 11th centuries" to a question about the Normans—feels abrupt. While template-based systems can mitigate this in narrow domains, they lack the flexibility required for general-purpose dialogue.
The goal of this work is to generate grammatically correct, fluent sentences from [Question + Factoid] pairs. This approach allows any extractive QA system to become more "natural" by simply appending this generation module.
Methodology: Dual-Encoder Pointer-Generator
The core challenge in this task is maintaining factual accuracy while ensuring linguistic variety. To address this, we employ a Pointer-Generator Network with a unique dual-encoder architecture.
Architecture Overview
Standard sequence-to-sequence models often struggle with Out-of-Vocabulary (OOV) words, such as rare names or technical terms. Our model overcomes this by using:
- Dual Encoders: Two separate 3-layered bidirectional LSTMs process the question and the factoid answer independently. This separation is vital because the syntactic relationship between a question and its answer is often indirect.
- Pointer Mechanism: The model calculates a generation probability ($P_{gen}$). It can either generate a common word from its fixed vocabulary or "point" to (copy) a word directly from the input sequence. This ensures that specific factual details like dates and names are preserved exactly.
Dataset: Scaling Natural Language Generation
To train this model, we developed a large-scale dataset of 315,000 [Question, Factoid, Full Answer] triples:
- Automatic Subset (~300,000): Generated by identifying sentences in source passages that contained the factoid and shared high overlap with the question.
- Manual Subset (15,000): Human-annotated samples providing complex sentence structures, including active and passive voice variations.
This hybrid dataset allows the model to learn the broad patterns of sentence restructuring from the large noisy set while refining its grammatical nuances from the high-quality manual set.
Key Results and Insights
Our evaluation shows that the system generalizes well across different domains, including knowledge bases (Freebase) and news articles (NewsQA).
- Metrics: The augmented model achieved a 74.05 BLEU and 86.25 ROUGE-L score on the manual test set.
- Context-Free Success: A significant finding is that the original source passage is not required for high-quality generation; the question itself contains sufficient structural information to rebuild a declarative sentence.
- Robustness: The neural approach significantly outperformed traditional rule-based templates, especially when dealing with multi-clause questions or varied syntax.
Conclusion and Future Directions
By decoupling answer generation from the source passage, we provide a flexible tool for enhancing the conversational quality of QA systems. While the model occasionally faces challenges with tense agreement and coreference resolution, future iterations utilizing Transformer architectures and explicit coreference modules promise even greater fluency and accuracy.
Summary updated