TL;DR
LLM agents often suffer from "over-exploration," continuing to invoke tools and search codebases long after they have gathered sufficient information to answer a user's question. JET (Just-Enough Exploration Tuning) is a novel fine-tuning method that trains agents to recognize this sufficiency, reducing tool calls by up to 33% and significantly lowering inference costs while maintaining or even improving answer quality.
What Problem Does This Paper Address?
In repository-level question answering (QA), agents must navigate massive codebases using tools like semantic search and file viewers. A major bottleneck is over-exploration: agents frequently continue to explore even after the relevant code has been found.
As shown in Figure 1, an agent might discover the correct version identifier in a setup.py file by the third step, yet continue for 26 more unnecessary tool calls. This is problematic for three reasons:
- Latency: Every tool call adds time.
- Cost: LLMs process the entire conversation history at every step; as the context grows, the cost per step increases superlinearly.
- Quality: Excessive context can actually confuse the model, leading to a lower final score than an earlier, "forced" answer.
Standard Supervised Fine-Tuning (SFT) often exacerbates this because it trains agents to strictly imitate "expert" trajectories, which themselves may contain redundant exploration steps.
Methods
JET introduces a three-stage pipeline to bake efficiency directly into the agent's policy:
- Golden Step Labeling: To find the optimal stopping point, the researchers force the agent to generate an answer at every step of a training trajectory. An LLM judge scores these intermediate answers. The "golden step" ($i^*$) is defined as the earliest step that achieves the maximum score.
- Rationale Insertion: The trajectory is truncated at the golden step. A stronger "rationale generator" model then writes a short explanation (a rationale) describing why the information gathered so far is sufficient to answer the question.
- Training with a Halting Weight: The agent is fine-tuned on these truncated trajectories. Crucially, JET uses a halting weight ($\lambda > 1$) to upweight the loss on the rationale tokens. This forces the model to pay more attention to the decision to stop and answer.
This approach differs from prior work by supporting free-form answers (rather than just binary "correct/incorrect" signals) and requiring no additional LLM calls at inference time.
Datasets
The authors evaluated JET using two primary repository-level QA benchmarks:
- SWE-QA-Pro: 260 questions focused on deep repository understanding.
- SWE-QA: A larger set of 720 questions.
Training was performed on two types of data: Self-generated trajectories (400 rollouts from the base models) and Expert trajectories (1,000 rollouts generated by Claude 3.5 Sonnet). The experiments utilized two open-weight 8B models: Qwen3-8B and Ministral-3-8B.
Key Insights and Ideas
- The Score Plateau: Figure 2 illustrates that for both base agents and experts, answer quality reaches a plateau early in the exploration process. Continuing to explore beyond this point provides diminishing returns or even negative utility.
- SFT is an Efficiency Trap: Standard SFT on expert trajectories successfully improves scores but often increases the number of tool calls because the student model learns to imitate the expert's exhaustive (and sometimes redundant) search patterns.
- Rationale as a Signal: Simply truncating trajectories isn't enough. The rationale acts as a "thought process" that helps the model internalize the logic of sufficiency, making the "stop" command more robust.
- Halting Weight is Critical: The authors found that the rationale is only effective when combined with the halting weight. Without upweighting, the "decision to stop" tokens are such a small fraction of the total trajectory that the model tends to ignore them.
Key Results
JET consistently outperformed both base models and standard SFT across multiple metrics:
- Reduced Tool Calls: JET reduced tool calls by 11% to 33% on self-generated trajectories and 18% to 28% on expert trajectories compared to SFT.
- Lower Inference Cost: By stopping earlier, JET slashed costs significantly. For Qwen3-8B on SWE-QA-Pro, the cost per 1,000 questions dropped from $19.2 to $8.5.
- Performance Preservation: Despite the drastic reduction in exploration, JET largely preserved (and sometimes improved) the task scores. In the "Expert" setting for Ministral-3-8B, JET actually achieved a higher score than standard SFT while using 20% fewer tool calls.
- Avoiding the Step Limit: As shown in Figure 4, JET agents reach the maximum step limit (30 steps) far less frequently than SFT agents, indicating they are successfully learning to conclude tasks autonomously.
Limitations and Future Work
- Domain Specificity: The study focused exclusively on Python repositories. Generalization to other programming languages or non-coding agent tasks remains to be seen.
- Judge Dependency: The "Golden Step" relies on an LLM judge. While common in current research, this introduces potential bias or variance from the judge model.
- Training Overhead: Labeling the golden step requires generating and scoring multiple answers for every step of every training trajectory, which is a significant one-time offline cost.
- Scaling: The researchers only tested 8B parameter models; future work could explore if these efficiency gains hold for much larger frontier models.
Summary updated