Abstract This research addresses the data scarcity challenge in Question Answering (QA) by leveraging large-scale, unlabeled Community-QA (CQA) data to automatically generate high-quality questions. By treating question generation (QG) as a dual task to answering, the authors develop a multi-stage engine that mines question patterns and predicts topics from source passages. Experimental results across SQuAD, MS MARCO, and WikiQA demonstrate that integrating these generated questions as an auxiliary signal significantly enhances the performance of state-of-the-art QA systems, particularly on datasets derived from real-world search logs.
The Data Scarcity Problem in QA
Modern Question Answering systems, whether based on knowledge bases or reading comprehension, rely heavily on large-scale, human-labeled QA pairs for training. However, manual labeling is prohibitively expensive and slow, leading to a scarcity of high-quality training data. The authors propose that question generation (QG) and question answering (QA) are two sides of the same coin: if a passage contains an answer, one should be able to "reverse-engineer" a relevant question from it. The challenge lies in generating questions that are grammatically correct, reflect common user intentions, and are grounded in the source text.
A Multi-Stage Question Generation Engine
The proposed system utilizes a multi-stage approach to transform passages into questions:
- Question Pattern Mining: Instead of generating text from scratch, the system mines "question patterns" (e.g.,
who founded # ?) from YahooAnswers. A clustering approach identifies common templates and their associated "historical topics"—the entities that typically fill the placeholders.
- Question Pattern Prediction: The authors explored two neural approaches: a retrieval-based CNN that ranks existing patterns and a generation-based RNN (BiGRU) with an attention mechanism that generates patterns word-by-word.
- Question Topic Selection: The system identifies a phrase or entity from the passage to fill the pattern placeholder. In the RNN model, this is handled via a "COPY mechanism" based on attention scores.
- Question Ranking: A linear model selects the final question based on prediction scores, language model fluency, and word overlap.
Dual Learning for QA Enhancement
A key innovation is the use of "Dual Learning" intuition to improve existing QA systems. When evaluating an answer candidate $A$ for a question $Q$, the system generates a set of questions from $A$. If these generated questions are highly similar to the original question $Q$, the system's confidence in the answer candidate increases. This creates a powerful auxiliary signal for ranking potential answers.
Experimental Results
The generation-based model (G-QG) consistently outperformed the retrieval-based model (R-QG), achieving higher BLEU-4 scores across all datasets (e.g., 13.57 vs 11.38 on WikiQA). More importantly, integrating these generated questions led to consistent gains in QA performance across all benchmarks.
The most significant improvements were observed on the MS MARCO dataset, which consists of questions from real search logs. As shown in the table below, the integration of QG signals (QA+QG) improved the Mean Average Precision (MAP) from 0.5131 to 0.5230.
Conclusion and Future Work
This work demonstrates that Community-QA sites are a goldmine for training QG models that reflect real user intent. While the system shows strong results, the authors identified areas for improvement, particularly in handling passages with multiple facts (40% of errors) and improving topic selection accuracy. Future work will focus on more robust independent models for topic selection and deeper exploration of the dual relationship between QA and QG.
Summary updated