A Double Phases Generation Network for Yes or No Question Generation (Student Abstract)

Jiayuan Xie, Feng Chen, Yi Cai, Zehang Lin

Abstract

Yes/No Question Generation (YNQG) is a critical task for automated student assessment, yet it remains challenging because the target answer ("yes" or "no") provides no semantic guidance for identifying relevant content in a passage. We present the Double Phases Generation Network (DPGNet), a novel architecture that addresses this by iteratively identifying and refining fact-related phrases. DPGNet utilizes a two-phase approach: an initial draft generation phase followed by a refinement phase using a gate attention mechanism. Our evaluation on the BoolQ dataset shows that DPGNet significantly improves question quality over standard encoder-decoder baselines.

Introduction

In AI-driven education, Question Generation (QG) systems typically rely on "answer-aware" frameworks, where a specific target answer (e.g., a named entity) anchors the question to a specific part of the text. However, Yes/No Question Generation (YNQG) lacks this anchor. Since the answer carries nearly zero information about the passage's content, models often struggle to determine which facts are salient. To bridge this gap, we propose a system that autonomously identifies and refines "fact-related phrases" to serve as anchors for generating high-quality boolean questions.

Methodology: The DPGNet Architecture

The Double Phases Generation Network (DPGNet) decomposes the YNQG task into two distinct stages: fact extraction and iterative refinement.

Architecture of the Double Phases Generation Network (DPGNet), illustrating the two-phase extraction and refinement process.

1st-Phase Prediction (Drafting)

In the first phase, a Bi-LSTM model scans the input passage to identify potential fact-related phrases using BIO (Beginning, Inside, Outside) tagging. These extracted segments are then processed by a standard encoder-decoder model with an attention mechanism to produce an initial "draft" question.

2nd-Phase Prediction (Refinement)

The second phase aims to improve the draft. A passage-to-question attention network uses the draft question to re-examine the original text, identifying more complete or relevant phrases. To mitigate the impact of noise—such as stopwords or irrelevant fragments—we introduce a gate mechanism. This mathematical filter controls the flow of extracted phrase information into the final generation stage. The refined features are then passed through the encoder-decoder again to generate the final yes/no question.

Experimental Setup

We evaluated DPGNet using the BoolQ dataset, a benchmark for boolean questions. The dataset consists of 12,697 samples (9,427 training, 1,635 development, and 1,635 testing), each containing a paragraph, a question, and a yes/no answer.

Results

DPGNet was compared against a baseline encoder-decoder ($NQG_{base}$) and a single-phase version ($NQG_{1p}$). We also included $NQG_{true}$, which uses human-annotated ground-truth phrases, as an upper bound.

Performance comparison of DPGNet against baseline models on the BoolQ dataset.

As shown in the results, DPGNet consistently outperformed the baselines across BLEU and ROUGE metrics. The improvement over $NQG_{1p}$ demonstrates the effectiveness of the iterative refinement and the gate mechanism in enhancing the linguistic quality and relevance of the generated questions.

Discussion and Future Work

The core innovation of DPGNet is the use of an initial draft as a "search query" to find better evidence within the passage. By treating YNQG as a self-guided fact-finding mission, the model overcomes the lack of semantic information in boolean answers.

However, several challenges remain. Generating "No" questions often requires logical manipulation of facts (e.g., changing a date), a process not explicitly handled by the current extraction-based approach. Additionally, the two-phase architecture increases computational overhead. Future research will focus on improving the efficiency of the generation process and exploring methods for more complex fact manipulation to create diverse and challenging assessment materials.

Summary updated

Abstract

This paper aims to solve the task of generating yes or no questions, which generates yes/no questions based on given passages. These questions can be used for evaluation automatically. We propose a double phases generation network that can identify specific phrases related to facts from the input passage and use them as auxiliary information for generation. Specifically, the 1st-phase prediction uses the extracted phrases as assistance to generate an initial question. Then, the 2nd-phase prediction utilizes an attention network to focus on the relevant phrases related to the initial question in the passage to generate questions that are more relevant to the specific facts contained in the initial question. Extensive experiments we performed on BoolQ dataset demonstrate the effectiveness of our framework.
BibTeX
@inproceedings{xie2021double,
  author = {Jiayuan Xie and Feng Chen and Yi Cai and Zehang Lin},
  title = {{A Double Phases Generation Network for Yes or No Question Generation (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.17962},
}
Figure