Modern Question Answering (QA) systems often struggle with "why" questions, typically retrieving long, redundant paragraphs rather than the concise, single-sentence answers required for voice assistants like Amazon Echo or Google Home. This paper, presented at AAAI-18, proposes a neural model designed to generate these "compact answers" by leveraging a novel architecture and a massive semi-distantly supervised dataset.
The Challenge of Compact Why-QA
Most open-domain QA systems rely on retrieval, returning relevant passages from a database. While effective for human readers, these passages often contain redundant context. For instance, in answering "Why did Yubari City go bankrupt?", a retrieved paragraph might repeat the question's premise (the "effect") multiple times.
Existing summarization methods often fail in this context because "why" answers are unique: the words in the question are usually redundant in the answer, yet the model must use the question to identify the specific "cause" within a multi-causal passage. Furthermore, standard span-based models like DrQA are insufficient because optimal answers often require generative capabilities to stitch together disparate words or adjust verb conjugations.
Methodology
The authors extend the Pointer-Generator Network—an architecture capable of both generating new vocabulary and copying words from the source—with two key innovations:
1. Question Word Marking (QWM)
The encoder is modified to "mark" words in the answer passage that also appear in the question. By feeding this overlap information directly into the LSTM, the model learns to distinguish between the "effect" (already known from the query) and the "cause" (the target answer). In Why-QA, question words often act as "negative signals" that should be excluded from the final compact answer to avoid redundancy.
2. Semi-Distantly Supervised Learning
To overcome the scarcity of human-labeled data, the researchers utilized a supervised causality recognizer to scan a 4-billion-page web archive. They extracted 101 million causality expressions to create a massive pre-training signal.
To teach the model to filter irrelevant information, they employed a "Virtual Passage" strategy. By merging three random causal relations into a single block of text, they forced the model to use the question as a filter to ignore the two irrelevant causes and extract only the correct one.
Experimental Results
The model was evaluated using a manual dataset of 19,639 triples and the auto-generated millions. The results demonstrated that the proposed method significantly outperforms both non-neural and neural baselines.
As shown in the table above, the "Proposed+250K" model (utilizing QWM and 250,000 auto-generated samples) achieved a ROUGE-L of 44.4, a substantial improvement over the base Pointer-Generator (36.0) and DrQA (23.4). Human evaluations further supported these findings, with the proposed model receiving a "Good" fluency rating 65.7% of the time, compared to 52.7% for the baseline.
Conclusion and Future Work
The study proves that semi-distant supervision is a viable substitute for expensive manual labeling in the Why-QA domain. While the model excels at generating fluent, compact answers, a remaining challenge is "informativeness." Human evaluators noted that without the original passage for context, many compact answers felt incomplete (only 44.1% were rated "Good" for informativeness in isolation). Future work may focus on improving the semantic completeness of these generated summaries and expanding the approach beyond the Japanese language.
Summary updated