When Memory Masquerades as Certainty: Grounded Uncertainty for Clarification Policies

Tool-using language-model agents are now trained to ask a clarifying question when a request is missing information, and to act directly when it is not. Every such method reads some internal signal of doubt, such as token entropy, semantic entropy, or a probability distribution over the tool's arguments. We argue that benchmark contamination, meaning test or training dialogues leaking into a model's training data, breaks this signal in one specific and predictable way. Suppose a model has seen a dialogue in which the user booked a flight to Lisbon on the 14th. When it later meets a version of that dialogue with the date removed, it can "remember" the 14th and fill it in with low entropy. The agent then executes the call instead of asking. The uncertainty-aware policy has not failed to estimate uncertainty. It has correctly reported that the model is confident, and the confidence comes from memory rather than from anything the user said.

This note proposes to show that this failure exists, to measure it under controlled contamination, and to fix it with a different uncertainty signal. The fix asks whether the model's belief about an argument value actually depends on the dialogue in front of it. We call this grounded confidence. It is measured by deleting dialogue turns one at a time and checking whether the model's probability for the value drops. A value that survives the deletion of every individual turn is not sourced from that turn. Values produced by memorization should behave this way, and the experiments will test whether they do. The agent acts only when the value is both likely and grounded. Otherwise it asks.

Core Claim

Our claim has two parts. First, contamination of tool-use dialogues causes memory-filled hallucination: when a required argument is removed, the model fills in the originally seen value with low uncertainty. The rate of this failure rises with the number of times the item was seen, and existing entropy-based clarification policies cannot detect it. Second, a leave-one-turn-out measure of grounding should distinguish ambiguous from fully specified requests about equally well on contaminated and clean items. Raw confidence should not.

What would be new is the conditional link between the two directions in the file. Contamination work measures memorization in order to correct leaderboards. Uncertainty-aware dialogue work assumes the model's confidence reflects the current context. Nobody has shown, or repaired, the point where the second assumption fails because of the first.

Relation to Prior Work

The uncertainty-aware dialogue line supplies the setting and the methods we compete against. Early POMDP dialogue managers already feed recognition confidence or entropy into the policy so it can fall back to confirmation. Examples are Spoken Dialogue Management Using Probabilistic Reasoning and Pietquin's Bayesian user model Réseau bayesien pour un modèle d'utilisateur…. Uncertainty Measures in Neural Belief Tracking… shows that calibrated belief trackers stabilize downstream policies. In all of these, uncertainty arrives from a noisy perception channel. In our setting, the misleading confidence comes from the model's own training history.

The closest modern work is SAGE-Agent, described in Structured Uncertainty guided Clarification for LLM Agents. It keeps a probability distribution over API argument values, uses the expected value of perfect information to choose which question to ask, and trains with uncertainty-weighted GRPO. GRPO, or Group Relative Policy Optimization, is a policy-gradient method that scores each sampled response against others drawn for the same prompt. SAGE's belief is read from the model's own distribution, so it inherits any memorized prior. Its benchmarks, ClarifyBench and When2Call, are fixed public scenarios, and the paper never audits them for leakage.

Two related methods modulate learning by the policy's confidence. EMPG amplifies updates on confident correct steps. SEED-GRPO downweights prompts with high semantic entropy. Both would reinforce memorized low-entropy behavior, which is the mechanism we expect to be harmful. Our proposal keeps SAGE's structured argument belief and replaces the quantity it gates on.

The contamination line provides the motivation and two key tools. Studying and improving reasoning in humans and machines finds near-zero surprise on memorized puzzles, which is direct evidence that exposure collapses uncertainty. Reasoning or Memorization? shows that reinforcement-learning gains on math benchmarks for Qwen2.5 come from retrieving memorized answers, and it introduces partial-prompt completion as a leakage audit. Qwen2.5 reaches 65.8% exact match on MATH-500 when given an 80% prompt prefix. We reuse that audit to stratify naturally contaminated items. Detecting RLVR Training Data via Structural Convergence of Reasoning detects training-set membership through the collapse of rollout diversity. We treat it as a competing gate rather than a solution, since it identifies that an item was seen, not which argument is unsupported.

ConStat and DyVal motivate our clean reference condition. ConStat compares performance on original versus rephrased benchmarks. DyVal generates problems from graphs on the fly, and on its tasks models that score well on static benchmarks fall to near zero. Both are evaluation-side remedies. Ours operates at decision time, inside the agent.

Finally, two papers constrain the design. When2Call shows that fine-tuning on abstention examples makes models "lazy," so any fix must be checked for over-asking. Can Tool-augmented LLMs be Aware of Incomplete Conditions? provides the ICT dataset (ddehun/ICT). Its "utterance removal" perturbation is exactly the operation that converts a memorized complete dialogue into an ambiguous one.

Research Program

We first build a controlled-contamination testbed. We take fully specified dialogues from ClarifyBench, When2Call, and ICT, split them into "exposed" and "unexposed" halves, and continue training an open base model on the exposed half. The model is Qwen2.5 at 3B and 7B, chosen because the file's contamination evidence concerns this family. We train at exposure counts of 0, 1, 4, and 16 passes, mixed into ordinary instruction data.

We then create ambiguous test variants of both halves by deleting the turn that supplies a required argument. Because the deletion is the same on both halves, any behavioral difference is attributable to exposure. Separately, we build a procedural generator of fresh tool schemas. It uses invented API names, randomized argument values, and randomized turn orders in the style of DyVal. This gives a reference condition that cannot be contaminated. Natural contamination in off-the-shelf models is assessed with partial-prompt completion on the public items.

Second, we define grounded confidence. Let $d$ be the dialogue, $d_{\setminus t}$ the dialogue with turn $t$ deleted, and $v$ a candidate value for argument $a$. The grounding score is

$$ G(v) = \max_{t} \big[ \log p(a{=}v \mid d) - \log p(a{=}v \mid d_{\setminus t}) \big], $$

which is the largest drop in the model's log-probability of the value caused by removing any single turn. The agent executes only if $p(a{=}v \mid d)$ and $G(v)$ both exceed thresholds tuned on the procedural clean set. Otherwise it asks about $a$. This costs one scoring pass per turn and requires no sampling. The method has a known weakness: a value stated twice survives single-turn deletion. We therefore also delete all turns that mention the value when a string or entity match exists, and we include a repeated-mention control set to measure what remains. In a training variant, we substitute grounded confidence for raw belief confidence in SAGE-Agent's uncertainty-weighted GRPO reward. This tests whether the signal can be internalized rather than computed at inference.

Third, we analyze. We track how memory-fill rate, raw entropy, and grounding change with exposure count. We also check whether grounded confidence over-asks on legitimate defaults the model knows from the world, such as a currency inferred from a country. A separate subset of such cases is constructed for this purpose.

Baselines

The naive reference point is the base model prompted with the tool schema and an instruction to ask when information is missing, with no uncertainty machinery. It is run on every condition so that each method's effect is measured relative to a common floor.

The strongest existing competitor is SAGE-Agent. It is the only method in the file that combines a structured argument belief with a learned ask-or-act policy, and it targets exactly our decision. Its reported numbers are 59.73% coverage on ClarifyBench ambiguous tasks with a GPT-4o base, using 1.39 questions per task. On When2Call, a 3B model trained with its uncertainty-weighted GRPO rises from 36.5% to 65.2% accuracy, per Structured Uncertainty guided Clarification for LLM Agents. Because the GPT-4o figure is not comparable to an open 3B/7B model, we reimplement SAGE on the same contaminated checkpoints. It receives the same training dialogues, the same rollout budget, and the same number of hyperparameter trials as our training variant. We first reproduce the reported 3B When2Call jump on the unexposed model as a sanity check.

We also run two further baselines. The first is When2Call's preference-trained model, which reaches macro F1 52.4 from 31.9 on Mistral-NeMo-Minitron 8B (When2Call). We apply its training recipe to our base model. The second is an entropy gate in the spirit of EMPG, which asks when step-level entropy is high, with thresholds tuned on the same clean set.

The ablation that isolates our key ingredient keeps everything about our policy fixed and replaces $G(v)$ with raw $p(a{=}v \mid d)$. Any gain therefore comes from the turn-deletion contrast alone.

We count our method as beating SAGE if it meets two conditions on items seen four or more times. First, it must close at least half of the gap in ask rate on ambiguous items between the contaminated model and the unexposed model. Second, it must not raise false asks on fully specified items by more than two points. These are our pre-registered targets, not reported figures.

Why not just…?

Why not just check whether the argument value appears in the dialogue text? For arguments copied verbatim from the user, this simple test may match our method, and that is a genuine threat. Many arguments, however, are inferred rather than copied. "Tomorrow" becomes a date, "my usual hotel" resolves from earlier turns, and spoken numbers get normalized. A memorized value can also coincidentally appear elsewhere in the dialogue. We will report results separately on an extractive subset and an inferred-argument subset. If string matching equals grounded confidence on both, we will drop the model-based score and recommend the string check.

Why not just detect contamination with Min-kNN and force the agent to ask on flagged items? Detecting RLVR Training Data via Structural Convergence of Reasoning reports an average AUC of about 0.70 for membership. That accuracy is modest, the detector needs dozens of rollouts per prompt, and it operates at the level of the whole item. On a seen dialogue whose arguments are all present, a membership gate would ask unnecessarily, which is the laziness When2Call warns about. We run this gate as an extra baseline, and the false-ask rate on exposed fully specified items will show whether item-level detection is too coarse.

Why not just use a verification checklist prompt? Structured Verification raises Gemma-3-4B F1 from 20.90% to 68.57% on ICT API Replacement (Can Tool-augmented LLMs be Aware of Incomplete Conditions?). However, its "sufficiency check" step is answered by the same model whose memory supplies the missing value. We expect it to degrade with exposure just as raw entropy does. We run it on every exposure level. If its memory-fill rate stays flat as exposure grows, the checklist is sufficient and our contribution shrinks to the diagnosis.

Why not just evaluate on fresh, generated dialogues and stop worrying? Generated tests fix the measurement problem that DyVal and ConStat address. Deployed agents, however, will meet user flows resembling their training data. The harm we describe happens in those familiar dialogues, which clean benchmarks deliberately exclude.

Proposed Experiments

The first decisive check tests the premise. On the contaminated checkpoints, memory-fill rate on ambiguous variants of exposed items should rise monotonically with exposure count, and raw entropy over the filled argument should fall. Memory-fill rate is the fraction of filled arguments equal to the deleted original value. On unexposed and procedural items, memory-fill should stay at the rate expected by chance. If exposure does not raise memory-fill, the proposal's motivation fails, and we will report that as a negative result about the robustness of these policies.

The second check compares signals. We measure the area under the ROC curve for separating ambiguous from fully specified items, using raw confidence, entropy, and grounded confidence in turn, at each exposure level. We predict raw confidence degrades with exposure while grounded confidence stays roughly flat.

The third check measures end-to-end behavior. For each baseline we report ClarifyBench coverage and questions per task, When2Call macro F1 and tool hallucination rate, and ICT F1. Each is split by exposed, unexposed, and procedural items, and by extractive versus inferred arguments. We also report false-ask rates on the repeated-mention and world-default control sets.

Off-the-shelf models stratified by partial-prompt completion rate provide a natural-contamination replication. Our prediction there is that memory-fill tracks completion rate. If the controlled and natural results agree, the claim holds for contamination as it actually occurs, not only for contamination we injected.