TL;DR
This paper moves beyond simple "failure mode" labels for LLM agents by introducing a trajectory-centered framework to trace how errors originate, propagate, and persist. The study reveals that faults enter early (median 18% into a run), and in 72% of failures, the evidence needed to catch the error was already present in the logs but ignored by the agent.
What Problem Does This Paper Address?
As LLM agents tackle increasingly complex, "long-horizon" tasks—like managing a simulated business or navigating multi-step software engineering problems—traditional failure analysis is falling short. Currently, when an agent fails, we usually just slap on a label like "task derailment" or point to the specific step where the crash happened.
The authors argue this is like diagnosing a plane crash by only looking at the crater. It doesn't tell us:
- Where the error actually entered: The "decisive step" identified by humans is often just the symptom of a wrong turn taken much earlier.
- How the error traveled: How a single false belief in a "fact sheet" contaminates the rest of the trajectory.
- Why it wasn't caught: Was the information missing, or did the agent simply ignore contradicting evidence?
Methods
The researchers developed a Trajectory-Centered Dependability Framework to perform a deep forensic analysis of agent failures.
Key components include:
- The Episode Record: A structured analytical record that links a candidate fault to its origin, propagation, detection, and eventual consequence.
- Two-Coordinate Mapping: Events are located by Operational Phase (e.g., Deliberation, Execution, Verification) and State Domain (e.g., Task State, Coordination State, Environment).
- Annotation Protocol: A 38-field protocol applied to 95 failed runs from multi-agent systems. The annotation was performed by an independent LLM (Claude) following strict rules to ensure consistency.
Datasets
The study analyzed failed trajectories from two prominent public corpora:
- Who&When: 184 failed runs involving systems like Magentic-One and CaptainAgent teams.
- MAD (Multi-Agent Diagnostics): 19 traces from various frameworks including MetaGPT and ChatDev.
- The final analyzed sample consisted of 95 failed runs after filtering for length and procedural validity.
Key Insights and Ideas
- Errors are "Early Birds": Faults enter very early in the trajectory (median at 0.18 of the run length), while the visible symptom doesn't appear until much later (median gap of 0.67 of the run).
- The Evidence is Hiding in Plain Sight: In 72% of failed runs, the trajectory already contained an observation or message that contradicted the erroneous belief before the failure occurred. In 88% of those cases, no agent acted on that evidence.
- State Contamination is the Real Killer: Most faults (71 out of 95) corrupt the "Task and Knowledge State" (R2). Once a false belief is written into a plan or fact sheet, it acts as a "poison pill" that persists even if the agent tries to "re-plan."
- The "Harness Tax": A surprising 45% of hand-crafted failures in the Who&When corpus were actually caused by provider content-filter crashes (HTTP 400 errors) rather than agent logic, yet these were often attributed to the agents themselves in the original dataset.
Key Results
The authors categorized the "Kind" of origin events to see what actually goes wrong at the start of a failure chain:
- Decision Faults (42%): The agent had adequate information but made the wrong choice.
- Fabrications (26%): The agent asserted a fact or tool result it never actually observed.
- Ineffective Recovery: While 31% of runs detected a deviation, zero of the resulting responses successfully removed the deviated state. Typical "re-planning" changes the next action but keeps the toxic belief that caused the loop in the first place.
- Control Reach: The study found that "Independent Judges" (who read the evidence the worker ignored) and "Environment Re-verification" would have reached the most episodes (74% and 43% respectively).
Limitations and Future Work
- LLM-as-Annotator: While the authors used a rigorous protocol and mechanical checks, they acknowledge the lack of human inter-annotator agreement in this version.
- Sandbox Constraints: The studied corpora were mostly question-answering or sandboxed programming. This meant "External Impact" (errors that change a real-world system in an undoable way) couldn't be fully explored.
- Counterfactuals: The study identifies which controls would have reached an error, but it doesn't re-run the agents with those controls in place to measure actual success rate improvements.
- Future Direction: The authors call for "Dependency-Guided Attribution" and better "Semantic Memory" controls that track the provenance and uncertainty of every stored fact.
Summary updated