TL;DR
"Cleaning" an AI agent's history by rewriting past mistakes as successes can backfire if the underlying environment hasn't actually been fixed. This study demonstrates that while truthful notes about errors help agents recover, falsifying the record to look "clean" often leaves the agent worse off than doing nothing at all.
What Problem Does This Paper Address?
Language model (LLM) agents are prone to making mistakes. As they work through complex tasks, their conversation histories—the "record"—accumulate these errors. To keep agents on track, researchers often use techniques to "clean" this history: compacting long contexts, repairing failed steps, or handing off a task to a fresh model.
However, there is a hidden danger: the record (what the agent thinks happened) can become decoupled from the world (the actual state of the database, files, or environment).
Prior work often assumes that if you "fix" a step in the history, the world is fixed too. This paper asks a critical question: What happens when the history lies? If a system tells an agent a task was completed successfully, but the error still persists in the world, does that "clean" history help or hurt?
Methods
The researchers employed a "frozen state" intervention. They paused agents mid-task, capturing both their exact history ($R$) and the exact state of the environment ($W$). They then manipulated either the record or the world before letting a "receiver" model (a fresh LLM) continue the task.
They used two distinct experimental designs:
- Fixed World, Varied Record (AgentCE-Bench): They kept a world state that contained a committed error. They then compared two ways of introducing the correct information into the history:
- Truthful Note: Appending a note that explicitly points out the error and provides the correct value.
- Falsified Record: Rewriting the agent's own previous failed action as if it had been successful with the correct value.
- Fixed Record, Varied World (AppWorld): They kept a "success receipt" in the history (claiming a database write worked). They then independently varied whether that write actually existed in the database ("effect present") or had been reverted ("effect absent").
Datasets
- AgentCE-Bench: A benchmark for tool-using agents where models must fill a grid by querying tools. It allows for fine-grained control over task difficulty and error detection.
- AppWorld: A more complex environment featuring 450+ APIs for everyday tasks (mail, banking, etc.) operating over a persistent database.
Key Insights and Ideas
- The "Sign Reversal": The most striking finding is that the same correct information can either help or hurt depending on how it's presented. On AgentCE-Bench, a truthful note helped the agent recover, but recording that same value as the agent's own "completed action" actually caused performance to drop below the unedited baseline.
- History as a Map: Agents treat their history as a map of "work remaining." If the history says a specific sub-task is done (even if it was a lie), the agent is highly unlikely to revisit that part of the world to check for errors.
- The World-Record Coupling: An edit's effect is fundamentally tied to the world it describes. A "success" message is a powerful boost if the action actually worked, but it becomes a major liability if the action failed.
- Presence of the Answer is Not Enough: Simply having the correct value somewhere in the context isn't a silver bullet. The contextual framing (e.g., "I did this" vs. "The user says this is wrong") drastically changes how the model utilizes that information.
Key Results
- AgentCE-Bench: Adding a truthful note about an error increased the "recovery rate" (the share of remaining targets the agent got right) from 51% to 66%. However, falsifying the record to show the action as completed lowered recovery to 44%.
- AppWorld: A "success receipt" in the history had a massive interaction effect. When the effect was actually present in the world, it raised success by ~10 points. When the effect was absent (the history was lying), it lowered success from 45% to 28%.
Limitations and Future Work
- Bundled Variables: The "truthful note" differs from the "falsified record" in several ways: the apparent author (user vs. agent), the placement, and the explicit mention of an error. Future work could disentangle these "role effects."
- Receipt Content: In the AppWorld experiments, success and failure receipts differed in their text content, not just their truth value.
- Automated Policies: This study used manual, controlled edits. Testing how automated history-cleaning policies (like LLM-based summarizers) behave when they accidentally introduce world-record disagreements is a vital next step.
- Model Specificity: While the results held across most high-end models (GPT-4o, Claude 3.5 Sonnet), smaller models like GPT-4o-mini showed less consistent behavior, suggesting that "trusting the record" might be a capability that emerges with scale.
Summary updated