Abstract
Large language models (LLMs) are increasingly expected to function as collaborative partners, engaging in back-and-forth dialogue to solve complex, ambiguous problems. However, current LLMs often falter in real-world settings, defaulting to passive responses or narrow clarifications when faced with incomplete or under-specified prompts—falling short of proactively gathering the missing information that is crucial for high-quality solutions. In this work, we introduce a new task paradigm: proactive information gathering, where LLMs must identify gaps in the provided context and strategically elicit implicit user knowledge through targeted questions. To systematically study and train this capability, we design a scalable framework that generates partially specified, real-world tasks, masking key information and simulating authentic ambiguity. Within this setup, our core innovation is a reinforcement finetuning strategy rewards questions that elicit genuinely new, implicit user information—such as hidden domain expertise or fine-grained requirements—that would otherwise remain unspoken. Experiments demonstrate that our trained Qwen-2.5-7B model significantly outperforms o3-mini by 18% on automatic evaluation metrics. More importantly, human evaluation reveals that clarification questions and final outlines generated by our model are favored by human annotators by 42% and 28% respectively. Together, these results highlight the value of proactive clarification in elevating LLMs from passive text generators to genuinely collaborative thought partners.
1. Introduction: The Problem of Information Asymmetry
Current Large Language Models (LLMs) often suffer from a "passivity trap." When presented with underspecified prompts, they typically exhibit two suboptimal behaviors: passive generation, where they guess missing details and produce generic content, or surface-level clarification, where they focus on trivial formatting details rather than high-leverage content.
This research addresses the issue of information asymmetry in human-AI collaboration. The goal is to transition LLMs from being mere text generators to proactive thought partners capable of identifying what they do not know and strategically eliciting that information from the user.
2. Methodology
The researchers framed proactive clarification as a Partially-Observable Markov Decision Process (POMDP). The approach relies on three core innovations:
- Synthetic Conversation Engine: To simulate real-world ambiguity, the team utilized existing detailed tasks and systematically masked "implicit" information (such as specific procedures or output requirements), leaving only the "explicit" objective visible to the model.
- Evidence-Sentence Reward: Rather than relying on sparse signals from final long-form outputs, the authors developed a reward mechanism based on whether a model's question successfully forced a "user oracle" to reveal specific, previously hidden sentences of information. This "curiosity proxy" provides immediate feedback for discovering high-value evidence.
- Reinforcement Fine-Tuning (RFT): A Qwen-2.5-7B model was fine-tuned using Proximal Policy Optimization (PPO), allowing the agent to explore various questioning strategies to maximize information gathering before generating a final response.
3. Datasets and Experimental Setup
The primary dataset used was DOLOMITES, consisting of 519 task templates across 25 professional domains, including medicine, law, and engineering. The data was split into visible "explicit" segments and hidden "implicit" segments. Additionally, the QMSUM meeting transcript dataset was used to train baseline models (like GPT-4o-Indirect) to evaluate if exposure to human brainstorming could naturally induce proactive behavior.
4. Key Results and Performance
The results indicate a significant leap in collaborative capability. The trained Qwen-2.5-7B-RFT model achieved a score of 0.65 on writing tasks, outperforming the much larger o3-mini (0.55) and GPT-4o (0.51).
Beyond automated metrics, human preference tests were equally telling. Human annotators preferred the clarification questions from the Qwen-RFT model 42% of the time over o3-mini. Furthermore, the final outlines produced after these proactive exchanges were preferred 28% more often than those from the baseline.
5. Discussion and Insights
Several critical insights emerged from the training process:
- The Necessity of RL: Supervised Fine-Tuning (SFT) on synthetic conversations actually degraded performance compared to base models. This suggests that simply mimicking "good" questions is insufficient; models require the trial-and-error of Reinforcement Learning to internalize the utility of specific information.
- Domain Sensitivity: Proactive gathering is most effective in open-ended fields like Social Science and Humanities, where tasks require deeper contextual reasoning compared to more procedural technical tasks.
- Efficiency: The model's performance generally peaked at approximately 5 turns of dialogue, demonstrating its ability to efficiently extract necessary information in a concise exchange.
6. Conclusion
By rewarding the discovery of information rather than just the final text, this framework successfully teaches LLMs to value the process of clarification. While limitations exist—such as the assumption of a fixed user intent—this work provides a scalable path toward creating AI agents that act as true collaborative partners in complex problem-solving scenarios.
Summary updated