Teaching Language Models To Gather Information Proactively

Tenghao Huang, Sihao Chen, Muhao Chen, Jonathan May, Longqi Yang, Mengting Wan, Pei Zhou

Abstract

Large language models (LLMs) are increasingly expected to function as collaborative partners, yet they often default to passive responses when faced with underspecified prompts. This work introduces a paradigm for proactive information gathering, where LLMs identify context gaps and strategically elicit implicit user knowledge. We present a scalable framework that generates partially specified tasks and employs a reinforcement learning strategy—specifically an Evidence-Sentence Reward—to train models to uncover hidden requirements. Our trained Qwen-2.5-7B model outperforms o3-mini by 18% on automatic metrics, with human annotators preferring its clarification questions and final outputs by 42% and 28%, respectively. These results demonstrate that proactive clarification is a key capability in transforming LLMs into effective thought partners.

Introduction: The Challenge of Information Asymmetry

In real-world human-AI collaboration, users often provide "lazy prompts"—vague or incomplete instructions that leave crucial domain expertise or specific requirements unspoken. This creates a state of information asymmetry. Standard LLMs typically respond to this ambiguity in two suboptimal ways:

  1. The "One-Pass" Fail: Attempting a final response immediately, which leads to generic or incorrect results due to missing context.
  2. Shallow Clarification: Asking generic questions (e.g., "Can you provide more details?") that focus on existing prompt content rather than identifying what is missing.

To address this, we propose moving LLMs from passive text generators to proactive information gatherers that can strategically elicit the "implicit" knowledge necessary for high-quality, specialized work.

Methodology: Training for Proactivity

We frame proactive information gathering as a Partially-Observable Markov Decision Process (POMDP) and introduce three core innovations to train models in this paradigm:

1. Task Masking Framework

To simulate authentic ambiguity, we take complex professional tasks and split them into Explicit Information (what a user typically says) and Implicit Information (the "hidden" domain expertise and specific requirements). During training, the model only sees the explicit portion and must learn to ask questions to uncover the masked implicit data.

2. Synthetic Conversation Engine

To scale training without constant human intervention, we utilize a "User Oracle" (a separate LLM). This oracle has access to the hidden information; the trainee model asks questions, and the oracle provides answers based strictly on the masked data, simulating a realistic back-and-forth exchange.

3. Evidence-Sentence Reward (RFT)

The core of our reinforcement learning strategy is the Evidence-Sentence Reward. Instead of rewarding the model for the stylistic quality of its questions, we reward their utility. Using Proximal Policy Optimization (PPO), a question receives a positive reward only if the oracle’s answer uncovers a specific sentence from the hidden implicit data that was not previously available. This discovery-driven reward ensures the model learns the value of the information it is seeking.

Experimental Setup and Datasets

Our primary training and evaluation used the DOLOMITES dataset, which contains 519 task templates across 25 professional domains (e.g., law, medicine, civil engineering). This dataset is unique for its expert-curated methodical steps. We also utilized QMSUM, a dataset of meeting transcripts, to provide indirect supervision for baseline models to learn natural human questioning patterns.

Key Results and Insights

Our trained 7B model, Qwen-2.5-RFT, demonstrated significant improvements over much larger state-of-the-art models:

  • Superior Performance: Qwen-2.5-RFT outperformed o3-mini by 18% on automatic evaluation metrics.
  • Human Preference: In head-to-head comparisons, human annotators preferred the clarification questions from the RFT model 42% of the time over o3-mini. They also preferred the final task outlines produced after the Q&A session by a margin of 28%.
  • Targeted Proactivity: Analysis showed that the RFT-trained model specifically learned to target "output requirements" and "procedures"—the exact types of information usually left out of underspecified prompts.

The Necessity of Reinforcement Learning

A critical finding was that standard Supervised Fine-Tuning (SFT) on synthetic dialogues actually hindered performance. While SFT can teach a model to mimic the structure of a conversation, it does not teach the model the value of the information it seeks. Reinforcement learning was necessary for the model to internalize the strategic importance of information gathering.

Domain Sensitivity

Proactive questioning proved most valuable in "open-ended" fields like the Social Sciences and Humanities, where user preference is paramount. In more procedural "Technology" tasks, the gains were present but less dramatic, as the path to a solution is often more standardized.

Conclusion and Future Work

This research highlights the value of proactive clarification in elevating LLMs from passive responders to collaborative thought partners. By rewarding the discovery of hidden evidence, we can train models to navigate ambiguity effectively. Future work will explore multi-turn strategies, where models ask a series of strategic questions to build complex mental models, and move toward "dynamic intent alignment" where the AI actively negotiates goals with the user over long-term collaborations.

Summary updated

Abstract

Large language models (LLMs) are increasingly expected to function as collaborative partners, engaging in back-and-forth dialogue to solve complex, ambiguous problems. However, current LLMs often falter in real-world settings, defaulting to passive responses or narrow clarifications when faced with incomplete or under-specified prompts, falling short of proactively gathering the missing information that is crucial for high-quality solutions. In this work, we introduce a new task paradigm: proactive information gathering, where LLMs must identify gaps in the provided context and strategically elicit implicit user knowledge through targeted questions. To systematically study and train this capability, we design a scalable framework that generates partially specified, real-world tasks, masking key information and simulating authentic ambiguity. Within this setup, our core innovation is a reinforcement finetuning strategy that rewards questions that elicit genuinely new, implicit user information -- such as hidden domain expertise or fine-grained requirements -- that would otherwise remain unspoken. Experiments demonstrate that our trained Qwen-2.5-7B model significantly outperforms o3-mini by 18% on automatic evaluation metrics. More importantly, human evaluation reveals that clarification questions and final outlines generated by our model are favored by human annotators by 42% and 28% respectively. Together, these results highlight the value of proactive clarification in elevating LLMs from passive text generators to genuinely collaborative thought partners.
BibTeX
@inproceedings{huang2025teaching,
  author = {Tenghao Huang and Sihao Chen and Muhao Chen and Jonathan May and Longqi Yang and Mengting Wan and Pei Zhou},
  title = {{Teaching Language Models To Gather Information Proactively}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=BbiAF8mSzT},
}
Figure