If you give a modern language model a sentence like "The declaration was signed in 1776," generating a question is trivial. The model can simply swap a few words to produce "When was the declaration signed?" This task requires zero historical context, zero logical inference, and zero conceptual understanding. It is a purely syntactic transformation—a surface-level shortcut.
For years, Automatic Question Generation (AQG) has been stuck in this surface-level trap. While models have become incredibly fluent, they frequently default to low-level factual recall. In educational settings, these questions fail to evaluate whether a student truly understands a concept or is merely matching keywords.
Recently, a distinct research direction has emerged to break this pattern. By shifting from simple word-swapping to structural composition, semantic mapping, and curiosity-driven objectives, researchers are developing systems capable of generating deep, high-cognitive-demand questions.
The Surface-Level Trap and Why Models Cheat
To understand why deep question generation is difficult, we must look at how traditional models learn to generate questions.
Most AQG models are trained on "answer-aware" datasets. In this setup, a model is given a text passage and a specific target answer span (usually a few words highlighted by a human wrapper). The model's job is to reverse-engineer a question for that exact span. As argued by Igor Labutov et al. in Deep Questions without Deep Understanding and echoed by Huanli Gong et al. in KHANQ: A Dataset for Generating Deep Questions in Education, this training paradigm heavily biases models toward shallow, sentence-level transformations.
When a model is trained primarily on direct text spans, it learns that the easiest statistical path to a valid question is surface-level pattern matching, as noted by Chien-Sheng Wu et al. in MixQG: Neural Question Generation with Mixed Answer Types. Andrea Horbach demonstrates in Linguistic Appropriateness and Pedagogic Usefulness of Reading Comprehension Questions that while this training produces grammatically flawless sentences, it lacks the information integration required for professional educational assessments.
The consequence is a major pedagogical gap. As highlighted by Xiaoqiang Wang et al. in SkillQG: Learning to Generate Question for Reading Comprehension Assessment, readers can answer these shallow questions through simple keyword matching without ever developing a conceptual understanding. Standard direct-answering metrics fail to isolate whether a student (or another AI model) possesses higher-order cognitive abilities or is simply exploiting surface-level cues, a limitation explored by Yuhang Zhou et al. in Teach2Eval: An Interaction-Driven LLMs Evaluation Method via Teaching Effectiveness.
[Shallow QG] Text Span ----(Syntactic Transformation)----> Factoid Question
│ (Solvable via keyword matching)
▼
Low Cognitive Demand
[Deep QG] Multi-sentence Context + Logic Chains ----> Conceptual Question
│ (Requires synthesis & inference)
▼
High Cognitive Demand
Forcing Depth via Structural and Semantic Anchoring
If end-to-end neural networks naturally default to the easiest surface-level shortcuts, how do we force them to generate questions that require deep reading? The answer lies in anchoring the generation process to underlying linguistic structures rather than flat sequences of words.
One approach is to force the model to look at the syntactic and relational skeleton of a sentence. Qingbao Huang et al. designed a dual-channel framework in Entity Guided Question Generation with Contextual Structure and Sequence Information Capturing that pairs linear text sequences with Graph Convolutional Networks (GCNs) to capture explicit syntactic structures. By forcing the model to navigate the structural dependencies between entities, it moves away from simple word rearrangement toward inferential inquiry. Similarly, Michael Flor et al. demonstrate in A Semantic Role-based Approach to Open-Domain Automatic Question Generation that leveraging Semantic Role Labeling (SRL) allows systems to systematically map out "who did what to whom," expanding the diversity and logical depth of both wh- and yes/no questions.
To move beyond single sentences and evaluate deep reading comprehension, models must connect dots across an entire text. Jun Araki et al. show in Generating Questions and Multiple-Choice Answers using Semantic Analysis of Texts that incorporating coreference resolution and paraphrase detection prevents keyword matching, forcing the question to rely on information distributed across multiple sentences. When dealing with narrative text, such as children's stories, Zhenjie Zhao et al. demonstrate in Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-centric Summarization that tracking event-centric summaries allows models to bypass surface facts and target high-cognitive-demand concepts like causal reasoning and character motivations.
Decomposing and Controlling Complexity
When a model attempts to generate a complex, multi-hop reasoning question in a single end-to-end pass, the generation often collapses or loses coherence. A key mechanism for achieving depth is compositionality—breaking a deep question down into explicit logical building blocks.
In Generating Commonsense Reasoning Questions with Controllable Complexity through Multi-step Structural Composition, Jianxing Yu et al. tackle this by generating simple "sub-questions" first, and then structurally composing them into multi-hop reasoning chains. This allows precise control over the question's difficulty and ensures that answering it requires external commonsense knowledge rather than simple text retrieval.
[Multi-step Composition]
Fact A (Text) --------> Sub-question 1 ──┐
├─(Structural Composition)─> Deep Commonsense Question
Fact B (External) ----> Sub-question 2 ──┘
This modular philosophy is equally vital for practical application. Xu Wang 0016 et al. argue in Towards Process-Oriented, Modular, and Versatile Question Generation that Meets Educational Needs that end-to-end models fail in classrooms because they lack pedagogical intent. By shifting to modular AI tools, instructors can guide the generation process step-by-step, moving from low-level factual recall to high-level conceptual questions that span entire syllabi.
When scaling this to massive documents, Mohammad Alizadeh et al. show in Savaal: Scalable Concept-Driven Question Generation to Enhance Human Learning that one-shot long-context prompting often defaults to shallow retrieval. Instead, using a concept-driven Map-Reduce approach to isolate and synthesize core ideas across a text yields quizzes that human experts judge to be significantly more effective at testing deep reasoning.
Modeling Curiosity, Gaps, and Reflection
The ultimate frontier in deep question generation is moving away from the text entirely. True human inquiry does not just reverse-engineer a statement; it arises from curiosity, knowledge gaps, and the desire to apply a concept to a new scenario.
To capture this, researchers are changing the fundamental objective of AQG datasets:
- Inquisitive Questions: Instead of asking about what is on the page, Wei-Jen Ko et al. introduce a framework in Inquisitive Question Generation for High Level Text Comprehension that models the forward-looking questions humans naturally ask to resolve knowledge gaps while reading.
- Information-Seeking Follow-ups: Real-world curiosity is iterative. As shown by the authors of FOLLOWUPQG: Towards Information-Seeking Follow-up Question Generation, tracking how humans dig deeper into a topic helps models move past answer-aware constraints toward creative, higher-order reasoning.
- Socratic Inquiry: In Socratic Question Generation: A Novel Dataset, Models, and Evaluation, Beng Heng Ang et al. pivot from answer-seeking entirely, training models to generate thought-provoking questions meant to challenge biases and prompt deeper reflection.
This shift from extraction to open-ended reflection changes how users interact with AI. For example, in Generating Reflective Questions for Engaging Gallery Visitors in ArtMuse, Sujatha Das Gollapalli et al. demonstrate how models can generate prompts that encourage users to relate content to their own subjective experiences.
Strikingly, keeping the model's focus away from immediate text retrieval can actually improve question quality. In developing an oral Q&A practice system, Mayuko Aiba et al. discovered in A ChatGPT-based oral Q&A practice system for first-time student participants in international conferences that large language models generate more insightful and challenging questions when they are not explicitly grounded in a reference paper upload. Pushing the model to operate at a conceptual level, rather than a retrieval level, naturally drives it to generate questions that demand deeper critical thinking.
Key Foundational Work
Good Question! Statistical Ranking for Question Generation (2010) — contributed 1 concept elements to this direction.
On Computational Sentence Generation From Logical Form (1980) — contributed 1 concept elements to this direction.