Language-model agents that call tools, browse, or run code fail in a predictable way. A small mistake early in a task, such as a wrong argument, a plausible but wrong tool, or a silently corrupted tool output, gets carried forward, and the agent keeps building on it. Recent systems add recovery machinery like rollback, local re-planning, and retry loops. Each of these machinery choices hides a decision the agent is bad at: how sure am I that things are still on track, and if they are not, which earlier step should I return to and what should I change? Agents are known to be overconfident in their own steps and to over-trust tool outputs. They are also over-skeptical once prompted to doubt. As a result, they recover too late, roll back too far, or flag benign variations as errors.
We propose to treat recovery as a decision made under calibrated probabilities. A small classifier reads the agent's hidden states. For each candidate pair of a resume point and a recovery action, such as retrying a call, changing its arguments, substituting a tool, re-planning from an earlier step, or asking the user, it estimates the probability that the task still succeeds. The agent then picks the pair with the best expected success net of cost. The claim is that calibration matters here specifically, not just better error detection. Comparing options that differ in cost and in how far back they go requires probabilities that mean what they say. A detector that only ranks well is not enough.
Core Claim
Prior work at this intersection either calibrates passively or recovers without calibrated probabilities. Uncertainty Calibration for Tool-Using Language Agents (PROBECAL) shows that a 3-layer MLP probe on hidden states predicts tool-execution success far better than token probabilities. It reduces expected calibration error (ECE, the average gap between stated confidence and actual accuracy) on TabMWP from 0.690 to 0.035 using only 500 training questions. However, it uses that probe only to pick the best of several pre-generated full rollouts, and it never intervenes mid-trajectory. Recovery systems such as BacktrackAgent, $R^3$DAO, and ReIn intervene mid-trajectory, but they rely on uncalibrated LLM judges or prompted reflection to decide when and how. $R^3$DAO explicitly names hallucinated diagnoses from cryptic logs as its main weakness.
Our proposal moves the probe from "which finished trace is best" to "which repair is worth its cost right now." Concretely, at step $t$ the agent chooses
$$ (j^\*, a^\*) = \arg\max_{j \le t,\ a \in \mathcal{A}} \Big[\hat p(\text{success} \mid h_t, j, a) - \lambda\, c(j, a)\Big], $$
where $h_t$ is the agent's hidden state at step $t$ and $j$ is the step to resume from. Choosing $j=t$ with action "continue" means doing nothing. $\mathcal{A}$ is the set of recovery actions, $\hat p$ is the probe's calibrated success estimate, $c(j,a)$ is the extra tokens and tool calls the option costs, and $\lambda$ is a user-set price per unit cost.
When costs are equal, only the ranking of $\hat p$ matters, and an uncalibrated detector would do as well. When costs differ, the decision depends on the actual size of the probability gaps. Rolling back five steps for a 3-point gain is a bad trade that an overconfident estimator will happily make. This yields a falsifiable prediction: the benefit of calibration over ranking alone should grow with $\lambda$ and with how unequal the option costs are.
Research Program
The probe needs labels of the form "from this state, resuming at step $j$ with action $a$, did the task succeed?" We generate them by counterfactual replay.
The procedure starts from trajectories of an open-weight agent, using Qwen2.5-7B and Qwen2.5-32B because hidden states are required. Errors are injected using the fault types that the literature shows agents miss:
- stochastic API timeouts and parameter errors, plus semantic drift (a plausibly named but wrong tool), from ResiliBench;
- silent numeric corruption of tool outputs, in the style of the Broken Calculator in Tools Fail;
- prompt-injection and response-corruption faults, following Aegis.
From each corrupted and each clean trajectory, we branch at every earlier step with every action in $\mathcal{A}$, run to completion several times, and record the empirical success rate. This is the counterfactual-replay idea that AgenTracer uses for offline failure attribution. We reuse it to supply supervision for an online controller rather than a post-hoc diagnosis.
The probe itself is a small MLP over pooled hidden states at the current step and at candidate step $j$, plus an embedding of $a$. It is trained with a proper scoring rule and then recalibrated with isotonic regression on a held-out split. Clean trajectories are included deliberately, so that "continue" is a well-calibrated option. This is how we target the over-skepticism problem that Tools Fail reports: once prompted to doubt, models reject benign deviations.
Two analyses accompany the system. The first measures how calibration of $\hat p$ degrades on tools and APIs held out from probe training. This is the open generalization question PROBECAL leaves. The second measures the false-intervention rate, meaning interventions on trajectories that would have succeeded untouched, as a function of $\lambda$.
Baselines
The naive reference is the same backbone agent with no recovery controller, reacting to environment error messages as it normally would. On ResiliBench the file reports Qwen2.5-7B at 63.5% success. Larger models score 67.7% (GPT-4o-mini) and 65.0% (Qwen2.5-32B) under optimal workflows, falling to 62.2% and 62.9% under flawed ones (ResiliBench). We rerun this reference ourselves on our splits rather than relying on these figures.
The strongest calibration competitor is PROBECAL-style trace selection Uncertainty Calibration for Tool-Using Language Agents. It generates N full rollouts and keeps the one the probe scores highest. It is the strongest competitor because it uses the same hidden-state signal and has the best reported calibration in the file, ECE 0.035 on TabMWP. We give it the same probe training data and set N so that its total token and tool-call budget matches our controller's average spend.
The strongest recovery competitor is an LLM-judge rollback loop in the style of BacktrackAgent. In this design, a verifier and a judge decide when to roll back, and a reflector proposes the fix. BacktrackAgent reports a 7.59% task-success gain over ReachAgent on Mobile3M at roughly half the inference speed. We implement the judge with the same backbone, give it the same action set $\mathcal{A}$, and tune its prompt on the same development split.
On τ-Bench we also compare against ReIn, which lifts ambiguous-scenario success from near 0% to 40–60% by injecting reasoning from an external monitor.
The key ablation keeps our probe and controller but removes recalibration and replaces $\hat p$ with raw probe scores. This baseline still detects and ranks options, but its probabilities are not calibrated. Beating it, and showing that the gap widens with $\lambda$, isolates calibration as the active ingredient.
Beating a baseline means higher task success at equal or lower cost, or equal success at clearly lower cost, together with a lower false-intervention rate. Each comparison is made at three values of $\lambda$.
Why not just…?
Why not just generate more rollouts and pick the best, as PROBECAL does? This is the most serious threat. Best-of-N avoids every mid-trajectory decision, and with large N it may match any controller. Our bet is that it wastes budget in long tasks, because each restart repeats the many correct early steps. $R^3$DAO's local repairs, for example, need about 104k tokens per task against roughly 3,000k for R&D-Agent on MLE-bench. If compute-matched best-of-N equals our controller on ResiliBench's Hard tier, we will report that calibrated rollback offers no advantage beyond budget allocation.
Why not just ask the model how confident it is, or use an LLM judge? Verbal confidence is unstable across prompts and tends to validate the model's own errors (Large Language Models Must Be Taught to Know What They Don't Know). Prompted humbling fails outright on some backbones; Vicuna-v1.5-7B stayed overconfident under every prompt in When Do LLMs Need Retrieval Augmentation?. The judge-loop baseline tests this directly, and we also report the ECE of the judge's own verbal confidence.
Why not just fine-tune the agent to recover, as AgentRefine does? AgentRefine trains on mistake-and-correction trajectories. It ends death loops and gains 13.3% on SciWorld over Agent-FLAN. However, it learns to react after an error becomes visible, and it carries no notion of cost or of silent errors. The two approaches are complementary: we will run our controller on top of an AgentRefine-style tuned policy. If our gains vanish there, recovery tuning already captures what calibration adds.
Why not just use a checklist before acting? Structured Verification raises Gemma-3-4B's F1 on ICT API replacement from 20.90% to 68.57% (Can Tool-augmented Large Language Models be Aware of Incomplete Conditions?). But it only gates the first invocation and says nothing about what to do after a failure, so we include it only as a pre-execution add-on to every arm.
Proposed Experiments
The main testbeds are as follows. ResiliBench's simulated tasks are split by API, so that 20% of the 30 APIs never appear in probe training. We use a Broken Calculator–style silent-corruption suite; no reported numeric result was found on Lacuna for the original Broken Calculator. We also use τ-Bench with ReIn's injected errors, and ALFWorld with perturbed actions.
Metrics are task success, tokens and tool calls per task, false-intervention rate on clean trajectories, ECE of $\hat p$, and regret relative to the oracle choice of $(j,a)$ computed from held-out counterfactual labels.
Three checks decide the outcome. First, the calibrated controller must beat the uncalibrated-ranking ablation, with the gap increasing as $\lambda$ rises. If the gap is flat, calibration is not the mechanism. Second, it must beat compute-matched best-of-N on long-horizon tasks. Third, ECE on held-out APIs must stay within a small factor of in-distribution ECE. If calibration collapses there, the method only works on familiar tools, and we would report that as the binding limitation.
References
- Uncertainty Calibration for Tool-Using Language Agents
- BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking Mechanism
- $R^3$DAO: Reactive Recovery and Reconstruction for Long-horizon Data Agent Orchestration
- ReIn: Conversational Error Recovery with Reasoning Inception
- ResiliBench: Evaluating Agentic Workflow Adaptation in Stochastic Environments
- Tools Fail: Detecting Silent Errors in Faulty Tools
- Aegis: Automated Error Generation and Attribution for Multi-Agent Systems
- AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?
- Large Language Models Must Be Taught to Know What They Don’t Know
- When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentation
- AgentRefine: Enhancing Agent Generalization through Refinement Tuning
- Can Tool-augmented Large Language Models be Aware of Incomplete Conditions?