Abstract
Multi-hop question generation (MHQG) requires reasoning across multiple text segments to formulate complex questions. Traditional approaches often rely on manually labeled supporting facts or utilize Recurrent Neural Networks (RNNs), which struggle with long-range dependencies and parallelization. This paper proposes a Transformer-based model for label-free MHQG. By integrating a pointer-generator mechanism, entity placeholders, and boundary marking, the model effectively processes multi-paragraph contexts. Experimental results on the HotpotQA dataset demonstrate that the proposed model significantly outperforms existing state-of-the-art label-free methods across all major evaluation metrics, including BLEU, ROUGE-L, and METEOR.
Introduction
Question Generation (QG) is a fundamental task in Natural Language Processing (NLP) with applications in education and automated dialogue systems. While standard QG typically focuses on information within a single sentence or paragraph, Multi-Hop Question Generation (MHQG) is significantly more complex, requiring the model to synthesize information across disparate text segments to form a cohesive inquiry.
Historically, MHQG models have faced two primary hurdles. First, many models depend on "supporting fact" labels—manually annotated sentences indicating relevant information—which are expensive and labor-intensive to produce. Second, previous label-free attempts have largely relied on RNN architectures. Due to their sequential nature, RNNs are difficult to parallelize and often fail to capture the long-range dependencies necessary for multi-hop reasoning across extensive contexts.
Methodology
The proposed model leverages the Transformer architecture, enhanced with several key mechanisms to address the specific demands of multi-hop reasoning:
- Pointer-Generator Mechanism: To handle Out-of-Vocabulary (OOV) tokens, such as rare names or technical terms found in Wikipedia, the model uses a copy mechanism. This allows it to either generate words from a fixed vocabulary or copy specific words directly from the source text.
- Entity Placeholders: Named entities are replaced with generic placeholders (e.g.,
[person_1]) during training. This encourages the model to learn underlying linguistic structures and entity types rather than becoming over-reliant on specific, rare names. Original names are restored during a post-processing step. - Boundary Marking: A special
[SEP]token is used to delineate boundaries between different context paragraphs, assisting the encoder in identifying cross-paragraph connections. - Enriched Embeddings: The input representation is augmented with Part-of-Speech (POS) tags and answer position tags (using BIO tagging), providing the model with additional structural and semantic context.
Experimental Setup
The model was evaluated using the HotpotQA dataset, which contains approximately 92,000 samples designed for multi-hop reasoning. The dataset includes "Bridge" questions, which link entities across paragraphs, and "Comparison" questions, which evaluate shared properties between entities.
Key Results
The proposed Transformer-based model was compared against MulQG, a leading RNN-based model for label-free MHQG. The results indicate a substantial performance gain across all metrics.
As shown in the table above, the proposed model achieved a BLEU-4 score of 19.42, representing a nearly 28% relative improvement over MulQG's 15.20. Similar gains were observed in ROUGE-L and METEOR scores, confirming the effectiveness of the Transformer architecture in handling multi-hop dependencies.
Discussion and Conclusion
The success of this model underscores the advantages of Transformers over RNNs for tasks involving long-range dependencies. By simplifying the task through entity placeholders and utilizing a pointer-generator, the model achieves state-of-the-art results without the need for complex graph networks or manual sentence labeling.
Future work will focus on scaling the model to even larger contexts, such as full Wikipedia articles, which may require more memory-efficient attention mechanisms. Additionally, incorporating human qualitative evaluations will be essential to further validate the logical coherence and naturalness of the generated questions.
Summary updated