When a human creates a question based on a specific piece of text, they begin with a clear target in mind. If the intended answer is "in 1969," the question naturally begins with When. If the target is "Marie Curie," it begins with Who.
In automated question generation (QG), however, standard sequence-to-sequence models frequently suffer from what researchers call interrogative mismatch or semantic drift. A model might generate a highly fluent, grammatically flawless question that asks Why or Where for a target answer that is a date or a person.
To resolve this, a prominent research direction focuses on answer-question semantic alignment: forcing generative models to tightly bind the linguistic intent of a question (its interrogative word) to the exact semantic category and boundaries of its target answer.
The Autoregressive Bottleneck
To understand why standard models fail, we must look at how they decode text. An autoregressive language model generates a question token by token, from left to right. The interrogative word (who, what, where, when, why, how) almost always appears at the very beginning of the sequence.
At this initial step, the model's hidden state is influenced by the entire source document. Because the model lacks an explicit mechanism to anchor its first few tokens to the specific semantic properties of the chosen answer span, it often defaults to the most frequent interrogative words in its training data or aligns with the broader document context rather than the specific target phrase. As observed by Sandeep Subramanian et al. in Neural Models for Key Phrase Extraction and Question Generation, this lack of tight coupling results in questions that are contextually relevant to the text but completely unanswerable by the selected answer phrase.
This decoupling leads to several distinct failure modes:
- Type Mismatches: Generating "where" for temporal answers or "why" for spatial answers (Samm Sun et al. in Answer-focused and Position-aware Neural Question Generation).
- Target Inclusion: Accidentally leaking the target answer inside the generated question itself (Liuyin Wang et al. in Answer-driven Deep Question Generation based on Reinforcement Learning).
- Semantic Inconsistency: Creating hallucinated question-answer pairs where the question does not logically lead to the provided answer (Dong Bok Lee et al. in Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs).
Strategy 1: Explicit Type Conditioning
The most direct way to fix the autoregressive bottleneck is to force the model to decide what kind of question to ask before it generates a single token.
Instead of letting the model guess the interrogative word implicitly, frameworks like the one proposed by Wenjie Zhou et al. in Question-type Driven Question Generation introduce a two-stage generation process. The model first predicts a discrete question type (e.g., who, location, reason) based on the semantic category of the target answer. This predicted type is then fed directly into the decoder as a hard constraint, guiding the initial token selection and preventing semantic drift.
[Target Answer: "Paris"] ──> [Type Predictor] ──> [Constraint: WHERE] ──> [Decoder] ──> "Where is the Eiffel Tower?"
In Knowledge-enriched, Type-constrained and Grammar-guided Question Generation over Knowledge Bases, Sheng Bi et al. apply a similar intuition to structured data. By constraining the decoder with explicit word-type predictions derived from knowledge base metadata, they ensure that the generated question's interrogative word strictly matches the semantic entity type of the target node.
Strategy 2: Structural and Positional Anchoring
Predicting the abstract type of an answer is often insufficient if the model loses track of where the answer is located or how it relates syntactically to the rest of the sentence.
To combat this, researchers incorporate structural and positional priors directly into the model's attention mechanisms. For example, Samm Sun et al. utilize relative distance modeling within the attention layer. This forces the model to focus its computational weight on the immediate linguistic context surrounding the answer span, preventing the decoder from copying irrelevant text from distant parts of the document.
Taking this structural grounding further, Qingbao Huang et al. develop a dual-channel framework in Entity Guided Question Generation with Contextual Structure and Sequence Information Capturing. They combine standard sequential representations with Graph Convolutional Networks (GCNs) that map the sentence's underlying dependency parse tree. By explicitly tracing the syntactic paths between entities and predicates, the model learns the exact grammatical role the answer plays, making the resulting question highly "answerable."
Similarly, Xiyao Ma et al. demonstrate in Improving Question Generation with Sentence-Level Semantic Matching and Answer Position Inferring that optimizing a model to simultaneously infer the answer's position and match sentence-level semantics drastically reduces the hallucination of irrelevant keywords.
Strategy 3: Closed-Loop Verification (Primal-Dual Learning)
A third highly effective intuition treats question generation and question answering as dual facets of the same underlying semantic relationship. If a generated question is truly aligned with a target answer, an independent question-answering (QA) system should be able to read the question and successfully extract that exact answer.
[Context + Answer] ──> [QG Model] ──> [Generated Question]
│
▼
[Target Answer] ◄─────────────────── [QA Model]
(Reward / Loss Signal)
This closed-loop verification is formalized by Qifan Wang et al. in Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word Generation. By integrating a QA module that "asks the question back," the framework evaluates whether the generated question uniquely isolates the target answer. If the QA model extracts a different span or fails to find an answer, a penalty is propagated back to the generator.
The authors of Answer-driven Deep Question Generation based on Reinforcement Learning implement this via reinforcement learning, using the answerability of the generated question as a reward signal. This explicitly penalizes the model for selecting incorrect interrogative words, ensuring that the question provides the exact semantic cues required to uniquely identify the target response.
Beyond Simple Entities
While aligning questions with distinct entities (like dates or locations) is well-understood, the frontier of this research direction addresses abstract, non-entity text spans.
When an answer is a clause, a reason, or a verb phrase, simple entity typing breaks down. As Heyan Huang et al. point out in Unsupervised Question Answering via Answer Diversifying, generating appropriate why or how questions for clausal answers remains a significant hurdle. To bridge this gap, researchers like Valentina Pyatkin et al. in Asking It All: Generating Contextualized Questions for any Semantic Role decouple the generation into semantic roles (prototypes) before contextualizing them into natural language.
Furthermore, in scenarios like closed-book question generation—where no supporting passage is present to act as a semantic bridge—the model must rely entirely on internal representations to align the question's intent with the long-form answer text (Xiangjue Dong et al. in Closed-book Question Generation via Contrastive Learning).
Ultimately, whether through explicit type constraints, syntactic graphs, or dual-task verification loops, the goal of answer-question semantic alignment remains the same: transforming sequence-to-sequence models from fluent text mimics into logically consistent, intent-aligned generators.
Key Foundational Work
Neural Models for Key Phrase Extraction and Question Generation (2018) — contributed 2 concept elements to this direction.
Question Generation for Question Answering (2017) — contributed 1 concept elements to this direction.
Answer-focused and Position-aware Neural Question Generation (2018) — contributed 1 concept elements to this direction.