Abstract: Large reasoning models (LRMs) have demonstrated remarkable proficiency in tackling complex tasks through step-by-step thinking. However, this lengthy reasoning process incurs substantial computational and latency overheads, hindering the practical deployment of LRMs. This work presents a new approach to mitigating overthinking in LRMs via black-box persuasive prompting. By treating LRMs as black-box communicators, we investigate how to persuade them to generate concise responses without compromising accuracy. We introduce Whisper, an iterative refinement framework that generates high-quality persuasive prompts from diverse perspectives. Experiments across multiple benchmarks demonstrate that Whisper consistently reduces token usage while preserving performance. Notably, Whisper achieves a 3x reduction in average response length on simple GSM8K questions for the Qwen3 model series and delivers an average ~40% token reduction across all benchmarks. For closed-source APIs, Whisper reduces token usage on MATH-500 by 46% for Claude-3.7 and 50% for Gemini-2.5. Further analysis reveals the broad applicability of Whisper across data domains, model scales, and families, underscoring the potential of black-box persuasive prompting as a practical strategy for enhancing LRM efficiency.
Introduction: The Bottleneck of Overthinking
Modern Large Reasoning Models (LRMs), such as DeepSeek-R1 and OpenAI’s o1, have revolutionized the field of artificial intelligence by mastering "Chain-of-Thought" (CoT) reasoning. While these models excel at complex problem-solving, they are prone to a significant efficiency bottleneck: overthinking. Even when presented with trivial tasks—such as basic arithmetic or simple logic—these models often generate excessively long reasoning traces. This behavior leads to high inference costs, increased latency, and a poor user experience for simple queries.
Existing attempts to curb this overthinking typically rely on supervised fine-tuning (SFT) or white-box interventions that require access to internal model weights. However, these methods are computationally expensive and inapplicable to closed-source proprietary APIs. Furthermore, simple instructional prompts like "be concise" often fail, as they frequently degrade the model's reasoning accuracy. This paper introduces Merlin’s Whisper (WHISPER), a "plug-and-play" black-box framework designed to persuade models into brevity without sacrificing logical depth.
Methodology: Persuasion as a Tool for Efficiency
WHISPER treats LRMs as human-like communicators that are susceptible to social and authoritative persuasion. Rather than using dry instructions, the framework appends iterative, persuasive prompt suffixes to the user's query. The framework operates through a three-stage process:
- Persuasive Prompt Creation: Using a generator model (e.g., GPT-4o), the system crafts suffixes from five distinct perspectives:
- Psychological (Emotion/Threat): Leveraging emotional appeals or high-stakes scenarios.
- Evidence-based: Citing authoritative (often synthetic) studies to justify brevity.
- Role-playing: Assigning the model a persona that demands speed, such as a coach in a timed competition.
- Detailed Instructions: Providing specific constraints on symbolic reasoning and token budgets.
- Candidate Evaluation: These prompts are tested on a development set (PDSet) and ranked based on their ability to minimize response length while maintaining accuracy within a strict tolerance.
- Iterative Refinement: The top-performing prompts serve as exemplars for the next generation, allowing the system to converge on highly effective persuasive strategies within a few iterations.
Key Insights and Experimental Results
Our evaluation across benchmarks like GSM8K, MATH-500, and GPQA-Diamond reveals several critical insights into LRM behavior:
- Social Susceptibility: Much like human subjects, LRMs respond more effectively to social or authoritative pressure than to literal commands. The "Evidence-based" approach, which cites fake research to support concise reasoning, proved to be one of the most potent strategies.
- Significant Token Savings: WHISPER achieved an average 40% reduction in token usage across all benchmarks. In the Qwen3 series, response lengths for simple GSM8K questions were reduced by a factor of three.
- API Compatibility: The method is highly effective for closed-source models. On the MATH-500 benchmark, WHISPER reduced token usage by 46% for Claude 3.7-Sonnet and 50% for Gemini 2.5-Pro.
- Cross-Model Generalization: Persuasive prompts optimized for smaller models (e.g., Qwen3-4B) often generalize to larger models in the same family, suggesting that model families share consistent "behavioral priors."
Conclusion
Merlin’s Whisper demonstrates that the efficiency of Large Reasoning Models can be significantly improved through black-box adversarial prompting. By understanding the "social" triggers of these models, we can effectively prune the "fluff" from their reasoning processes. This approach offers a practical, low-cost strategy for deploying high-performance reasoning models in latency-sensitive applications. Future work will explore the application of WHISPER to open-ended dialogue and creative writing domains, as well as its interaction with the latest generation of "o1-style" models.
Summary updated