Don't Forget What You Just Learned: Self-Directed Repair Turns Agent Mistakes Into Lasting Lessons

Yuval Kansal, Karen Ullrich, Polina Kirichenko, Ilia Kulikov, Niraj K. Jha, Mohammad Pezeshki, Mark IbrahimarXiv 2026 · arXiv

TL;DR

AI agents often "forget" the lessons they learn from mistakes once a session ends, leading to costly repeated errors. Self-Direction solves this by allowing agents to reflect on their failures and distill lessons into tiny, modular "repair adapters" (LoRA) that are stored in a library and automatically reused for similar future tasks, enabling continuous improvement without the risk of overwriting the base model's core capabilities.

What Problem Does This Paper Address?

Modern AI agents are surprisingly good at correcting themselves during a task if they hit a snag. However, this learning is ephemeral. Because the underlying model weights remain unchanged, the agent is just as likely to make the exact same mistake in a brand-new session.

Current workarounds have major drawbacks:

  1. In-context learning: Shoving "lessons" into a prompt works temporarily, but context windows are limited and expensive. You can't keep every past mistake in the prompt forever.
  2. Full fine-tuning: Updating the entire model is computationally heavy and risks "catastrophic forgetting," where the model learns to fix one bug but breaks three other features in the process.

The authors tackle the challenge of creating a persistent, scalable, and safe way for agents to learn from their own "failure traces" across diverse domains like coding, tool use, and web browsing.

Methods

The researchers introduced a loop called Self-Direction, which follows a three-step process:

  1. Self-Reflection: When an agent fails a task (detected via environment feedback or a verifier), it examines its own failed trajectory. It acts as its own teacher to synthesize a small set (up to 24) of "procedural" instruction-response examples that show the correct way to handle that specific failure.
  2. Distillation into LoRA: These examples are distilled into a lightweight Low-Rank Adaptation (LoRA) adapter. To ensure stability, they use a novel SVD-based initialization: they freeze one part of the adapter (Matrix A) to anchor it to the model's most important features and only train the other part (Matrix B). This results in a tiny "patch" (only ~0.02% of the base model's parameters).
  3. Library & Routing: These patches are saved in a "library of lessons." When a new task comes in, a router compares the task description to previous failures. If it finds a match, it loads the relevant adapter. If the adapter-guided attempt fails, the system unloads it and falls back to the base model.
The Self-Direction pipeline: from mistake detection to adapter training and future reuse.

Datasets

The authors evaluated Self-Direction using open-weight models (specifically the Qwen family) across several rigorous benchmarks:

  • GAIA: General assistant tasks involving web search and file manipulation.
  • Claw-Eval: Complex tool-use workflows (email, calendar, task tracking).
  • Terminal-Bench 2.1: Hard, realistic coding and system administration tasks in a Linux shell.
  • MiniWoB++: Browser-based interaction tasks.

Key Insights and Ideas

  • Failure as a Signal: Instead of needing a "gold standard" teacher, the agent uses its own failure as the training signal. This "teacherless" approach is highly scalable.
  • Modular Memory is Better than Context: The paper demonstrates that storing lessons in weights (adapters) is far more efficient than text-based "Skills.md" files. As mistakes accumulate, text files eventually overflow the context window, whereas a library of adapters can grow almost indefinitely.
  • Lessons Generalize: This is the "aha moment." A lesson learned from a failed "ORCID publication search" can actually help the agent succeed at "Wikipedia revision counting." The adapters aren't just memorizing answers; they are learning procedures like pagination, data verification, and budget management.
  • Safety through Freezing: By keeping the base model weights frozen and using isolated adapters, the agent's core intelligence is never "corrupted" by the learning process.

Key Results

Self-Direction consistently turned failures into successes across all benchmarks:

  • Performance Lift: It improved success rates by 17.0 percentage points on GAIA and 13.8 points on Terminal-Bench.
  • Beating the Giants: A 27B-parameter Qwen model equipped with Self-Direction outperformed GLM-5.1 (a model 30x larger) and performed competitively with frontier closed-source models like Claude 4.7 and Gemini 3.1 Pro.
  • Reliability: It nearly doubled the agent's reliability across repeated encounters of the same task. While base agents often fluctuate between success and failure, Self-Direction "locks in" the fix.
Self-Direction outperforms standard retrying and in-context reflection on GAIA.

Limitations and Future Work

  • Mistake Detection: The system currently relies on external feedback (like a code test failing or a user saying "that's wrong") to trigger the learning loop. If a mistake goes unnoticed, no lesson is learned.
  • Compute Overhead: While the adapters are tiny, the process of reflection and training takes about 8 minutes on an H200 GPU. This makes it a "post-failure" background process rather than an instantaneous update.
  • Model Specificity: Because the lessons are distilled into weights, an adapter trained for Qwen cannot be used by a Llama model.
  • Future Directions: The authors suggest exploring how to compose multiple adapters at once to solve complex tasks that might require several different "lessons" simultaneously.

Summary updated

Abstract

Intelligent agents should learn from their mistakes so that experience leads to continuous improvement. Yet today's agents recover from a mistake only temporarily: in-context reflections are easily forgotten across sessions and compete for limited context, while full-parameter fine-tuning is expensive and can interfere with existing capabilities. We introduce Self-Direction, a mechanism for agents to learn from mistakes that combines the benefits of temporary in-context retries and policy updates. When an agent fails a task, it reflects on its own trajectory in order to extract a lesson and immediately distills it into a lightweight low-rank adapter. By keeping the original weights frozen, Self-Direction prevents adapter training from overwriting base-model parameters while training additional parameters equivalent to ~0.02% of the base model's parameter count for each adapter. The agent then saves these adapters in a persistent library of lessons, automatically routing them to solve related problems later. Experiments with agents based on open-weight models from the Qwen family across open-ended research (GAIA), tool-use workflows (Claw-Eval), and terminal tasks (Terminal-Bench 2.1) show that our method improves overall success rates by 8.9 to 17.0 percentage points and performs competitively with frontier systems. We find that Self-Direction reduces repeated mistakes across future encounters of a task, nearly doubling the reliability of an agent's corrections. Learned lessons also transfer: a handful of adapters generalize to dozens of new tasks without additional training. Ultimately, this self-directed repair transforms everyday mistakes into lasting lessons.
BibTeX
@misc{kansal2026don,
  author = {Yuval Kansal and Karen Ullrich and Polina Kirichenko and Ilia Kulikov and Niraj K. Jha and Mohammad Pezeshki and Mark Ibrahim},
  title = {{Don't Forget What You Just Learned: Self-Directed Repair Turns Agent Mistakes Into Lasting Lessons}},
  year = {2026},
  eprint = {2610.31846},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.31846},
}
Figure