When the world changes, the cheapest way to update a vision-language model (a model that answers questions about images in text) is to put the new fact in its prompt: "As of 2024, this stadium is home to team X." This is in-context learning, meaning the model adapts from text placed in its input with no change to its weights. It works well for direct questions. The trouble starts with questions that need several reasoning steps. If the model writes out its reasoning first (chain-of-thought), the final answer can follow the new fact while one middle step still relies on the old fact stored in its weights. The model can also do the reverse. The chain still reads fluently, and if it comes with boxes drawn on the image, it looks well supported. Human-subject work suggests readers rarely notice a quietly missing or stale premise. This proposal treats that silent stale step as the thing to measure and catch.
We propose to check, for each step of a model's written reasoning, where that step actually comes from. A step can come from a region of the image, from the in-context update, or from the model's stored memory. We test this by intervention rather than by asking the model. For each step, we remove or alter one candidate source (blur the boxed region, delete the update, or swap in a contradicting update) and measure how much the model's probability of writing that step changes. Steps that change with none of the visible sources are relying on memory. When such a step concerns a fact that changes over time, we flag it as possibly stale. The question is whether these intervention-verified labels predict wrong multi-hop answers, and help people catch them, better than self-reported citations, model confidence, or duration-based staleness scores.
Core Claim
After an in-context knowledge update, vision-language models produce a measurable rate of chains in which the answer and at least one middle step disagree about which version of a fact holds. We call these stale steps. Self-reported source tags will not reliably identify them. Intervention-based source checks will, and the resulting flags will improve both automatic error prediction and human error detection on multi-hop questions about edited entities. If cheap self-tagging turns out to track the interventions closely, the method collapses to a prompt, and we will report that.
What Prior Work Establishes and Leaves Open
Three lines of work set up the problem without solving it.
The first is knowledge editing for vision-language models. VLKEB and KEBench benchmark editing methods on real knowledge-graph images, scoring reliability (getting the edited fact itself right), generality (rephrased questions or other images), locality (leaving unrelated knowledge intact) and portability (using the edited fact in 1- to 4-hop reasoning). They find that prompt-based editing, In-Context Knowledge Editing (IKE), ports best, but portability decays with hop count. They score only final answers and never inspect reasoning. CARML keeps edits reliable over long edit streams using multimodal retrieval, learned soft prompts and logit correction. It outputs short answers with no rationale. These papers motivate the setting and supply the data, and none looks inside the chain.
The second is reasoning under in-context edits. DecKER shows that in text-only models, conflicting injected facts can wreck multi-hop reasoning (reported drops above 80%). It repairs this by having the model first write a reasoning outline with placeholders and then fill each placeholder with the injected facts. This is the closest existing method. It is text-only, it never grounds steps in image regions, and it prevents conflicts by construction rather than detecting which steps in a free-form chain went stale. We port it to images as a baseline, and our checks can run on top of it.
The third is grounded justification and its human effects. PostAlign trains models to emit boxes and masks alongside chain-of-thought. It finds that language priors override correct early-layer visual evidence in the final layers, and grounding raises LLaVA-1.5-7B POPE accuracy from about 80% to over 84%. It assumes static knowledge. Attentive Explanations and Faithful Multimodal Explanation for VQA established pairing text explanations with pointing, and the latter used gradient checks to filter unfaithful explanations. Our intervention test borrows that faithfulness logic and moves it from training data to individual inference-time steps.
Two human studies point in opposite directions, and we take the conflict seriously:
- Fool Me Once? finds that adding visual maps to text helps clinicians catch errors.
- Critical or Compliant? finds that omissions are the most common error in frontier-model chains (36–42% of GPT-4o errors) and the hardest for people to spot. Contradictions are caught 82% of the time, and confident tone lowers detection by up to 10 points.
A stale step resembles an omission more than a contradiction, because the updated premise is simply not used. So boxes alone may make things worse. Our human study tests whether explicit source flags turn stale steps into something people can see.
On the temporal side, Mitigating Temporal Misalignment by Discarding Outdated Facts predicts how long a fact stays true and discounts confidence when the evidence is likely expired. It reports a 50–60% reduction in calibration error on SituatedQA. We reuse its duration predictor to decide which memory-sourced steps deserve a flag. RealTime QA documents the "outdated trap," where models fall back to stored facts when retrieval is weak. Our flags are meant to expose exactly that fallback when it happens inside a chain.
Research Program
Measurement. We take VLKEB portability cases, whose knowledge-graph paths tell us which hop depends on the edited fact. We prompt LLaVA-1.5-7B and Qwen-VL, both evaluated in VLKEB, with the image, the in-context edit, and a request for a step-by-step answer that marks the image region each visual step relies on. An automatic judge, validated on a hand-labeled sample, aligns each written step to a gold hop and labels it as asserting the new fact, the old fact, or neither. The stale-step rate is the fraction of chains whose edited hop asserts the old fact. We report it separately for chains with correct and incorrect final answers, and by hop count. This answers whether the text-only entanglement failure documented by DecKER also appears in grounded multimodal chains.
Source checking. For step $s_i$ and candidate source $c$, we compute a sensitivity score under teacher forcing (fixing the chain's earlier tokens so only the intervention varies):
$$\Delta_i(c) = \frac{1}{|s_i|}\left[\log p(s_i \mid \text{full input}, s_{<i}) - \log p(s_i \mid \text{input with } c \text{ ablated}, s_{<i})\right]$$
Here $|s_i|$ is the step's token count and $s_{<i}$ the preceding steps. The ablation is region blurring for an image source and deletion or contradiction-swap for the update source. A step whose sensitivity to every visible source falls below a threshold, set on held-out data, is labeled memory-sourced. A memory-sourced step about a fact with a short predicted duration is flagged.
The cost is roughly two to three extra forward passes per step. We report this overhead, since grounding overhead is already a known limitation of multi-output justification.
Human study. We follow the perturbation-and-detection design of Critical or Compliant?, applied to factual edited-entity questions. Participants see the answer, the chain and the boxes, under three conditions: no flags, self-reported tags, or intervention flags. We also cross tone (confident or neutral), because confident delivery is a known confound.
Secondary analysis. We ask whether compressing the edits into activation vectors changes source behavior. Following Multimodal Task Vectors, we replace many in-context edits with patched attention-head activations and test whether chains still show update-sensitive steps. If they do not, compression saves context but makes the update invisible to source checking. The task-vector literature has not tested this; In-Context Learning Creates Task Vectors explicitly leaves multi-step outputs open.
Baselines
Plain IKE. The naive reference point is IKE with direct answering and no chain, as evaluated in VLKEB, which reports it as having the highest portability among tested editors. No numerical IKE portability value was found on Lacuna; the file reports only that parameter-update methods are "often below 50%" and that roughly 60% one-hop accuracy degrades by the fourth hop. We rerun IKE ourselves on the same evaluation split. A second naive reference is IKE with free chain-of-thought and grounding but no source checking. This measures whether reasoning alone helps or hurts portability.
Multimodal DecKER port. The strongest existing competitor for answer accuracy is DecKER ported to images. The model writes a placeholder outline, the first placeholder is filled by a grounded visual recognition step, and later placeholders are filled from the edits. Its best reported number is 59.0% multi-hop accuracy on MQuAKE-CF-3k-v2 with Llama-3.1-8B, against 16.8% for PokeMQA. That is a text benchmark, and no multimodal DecKER result was found on Lacuna. We run it with the same models, prompts budget, Best-of-N sample count and judge as our method. It is strongest because it attacks stale steps by construction. If its stale-step rate is already near zero, detection has little left to do.
Error prediction baselines. Our flags are scored as predictors of a wrong final answer (area under the ROC curve) and of a gold-labeled stale step (precision and recall). We compare against three alternatives under matched calls to the duration predictor: the model's own answer confidence; the duration-discounted confidence of Mitigating Temporal Misalignment, whose reported adaptive-inference result is 20.5 exact match on SituatedQA (text, N=322); and self-reported source tags.
Ablations. These isolate the key ingredient by removing interventions on one source at a time (image, update), and by using deletion versus contradiction-swap ablations.
What counts as winning. On automatic metrics, intervention flags must beat self-tags and duration-discounted confidence on stale-step recall at matched precision, by a margin outside bootstrap intervals. In the human study, intervention flags must raise detection of stale-step errors over the no-flag condition without lowering acceptance of correct chains.
Why not just…?
Why not just fine-tune the model on the new facts? In the file's evidence, parameter edits fail exactly where this proposal operates. MEND drops below 1% reliability after 1,000 sequential edits on LLaVA-1.5 (CARML), produces NaN values after a few dozen edits (VLKEB), and fine-tuning ports poorly to multi-hop questions. We include fine-tuning only as a reference row in the measurement study, to show whether its chains go stale even more often.
Why not just use retrieval-augmented generation and trust the retrieved text? Retrieval is the update channel here, so this is not an alternative so much as the setting. RealTime QA shows that 34% of errors come from retrieval failures, after which models fall back to stored facts. DecKER shows that prepending conflicting facts can break multi-hop chains. Retrieval decides what goes in the prompt. It does not tell anyone which step ignored it, and that is the gap we measure.
Why not just prompt the model to cite its source for each step? This is the real threat, because it costs nothing extra. Self-citation is a model output and can be as unfaithful as the chain itself. PostAlign's evidence that late layers override earlier visual evidence suggests self-reports will drift toward plausible-sounding citations. The self-tag baseline and its agreement rate with the interventions settle the question directly. If agreement is high and self-tags match intervention flags on error prediction, we will recommend the prompt and treat the interventions as its validation.
Why not just use DecKER-style placeholder reasoning so stale steps never occur? If the multimodal port drives stale steps near zero while keeping accuracy, that is a strong result, and our contribution narrows to the measurement study. We expect residual failures to remain, because the visual recognition hop cannot be placeholdered: it must come from the image, and the model's memory of how an entity looks can itself be outdated. We report stale-step rates under DecKER and apply our flags to its chains to see what they still catch.
Decisive Checks
The project hinges on three results.
The first is whether stale steps exist at a non-trivial rate in grounded chains on VLKEB 2- to 4-hop cases, especially chains with correct final answers. If the rate is negligible, the problem is not real for these models and we stop.
The second is whether intervention flags beat self-tags and duration-discounted confidence at predicting stale steps and wrong answers. We also apply the same pipeline to MMEdit E-VQA, as a check that the flags' advantage is not specific to VLKEB's knowledge-graph structure.
The third is the human study: whether flags close the gap between how often stale-step errors occur and how often people detect them, measured against the omission-detection pattern reported in Critical or Compliant?, and whether confident tone still erodes the benefit.
The secondary result, whether activation-compressed edits lose update sensitivity, decides whether compression-based multimodal editing can be paired with checkable justifications at all.
References
- VLKEB: A Large Vision-Language Model Knowledge Editing Benchmark
- Reliable Lifelong Multimodal Editing: Conflict-Aware Retrieval Meets Multi-Level Guidance
- Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editing
- PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs
- Attentive Explanations: Justifying Decisions and Pointing to the Evidence.
- Faithful Multimodal Explanation for Visual Question Answering
- Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting
- Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations
- Mitigating Temporal Misalignment by Discarding Outdated Facts
- RealTime QA: What's the Answer Right Now?
- Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learning
- In-Context Learning Creates Task Vectors