A Matrix with Steps: An Episodic and Procedural Memory Method for Tool-Using Agents and Robots

Jürgen Paulhart, Peter SchüllerarXiv 2026 · arXiv

TL;DR

This paper introduces a hybrid memory system for robots and AI agents that combines a fast associative matrix "finder" with robust, ordered step records. By enforcing a "write gate" that only accepts externally verified successes—rather than an agent's own self-assessment—the method prevents error propagation and significantly boosts task success rates in complex environments.

What Problem Does This Paper Address?

Current AI agents and robots suffer from two major "memory" flaws: they often repeat actions that previously failed, and they frequently "hallucinate" success. Most modern Large Language Model (LLM) agents rely on self-reflection to decide if a task was successful. If the model thinks it succeeded when it actually failed, that failure becomes a "gold standard" example in its memory, leading to a cycle of reinforced errors.

Furthermore, many memory systems store experience as raw text or long trajectories. This makes it difficult to extract general procedures, handle "right to erasure" requests (deleting specific user data without breaking the whole model), or transfer a "lesson" learned by one robot to another with a different physical build.

Methods

The authors propose a two-part memory architecture:

  1. The Associative Matrix (The "Finder"): This is a linear associative memory updated using the delta rule. It acts as a high-speed lookup table. Given a goal, it finds the right procedure; given a current step, it predicts the next one. Because it uses superposition (layering information), it is incredibly fast but cannot "unlearn" a single item without rebuilding.
  2. Ordered Step Records (The "Source of Truth"): Since matrices lose order and are hard to edit, the actual procedures are stored as addressable records. These records hold the sequence of goals (e.g., "pick up the bolt") rather than specific motions (e.g., "move joint A by 10 degrees"). This abstraction allows the same procedure to be used by different robots.

The "Write Gate": This is the system's most critical policy. An episode is only recorded as a "success" if verified by an external check—a software test, a tool return code, or a human verdict. The agent’s own claim of success is never trusted.

Datasets

The researchers evaluated their system using a mix of simulation and synthetic benchmarks:

  • MuJoCo Simulations: They used a planar two-link arm and a UR5e industrial arm to test task execution and "hidden" success criteria (like arrival speed).
  • Synthetic Episodes: A dataset of 100,000 synthetic episodes (457 MB) was used to test lookup speed and retrieval accuracy.
  • Real-World Text: 3,000 procedures were extracted from an existing agent platform and embedded using models like bge-m3 and Qwen3-Embedding-8B to test the matrix's capacity to handle real-world language.

Key Insights and Ideas

  • Goals over Trajectories: By storing "what to do" (procedural goals) instead of "how to move" (joint trajectories), the memory becomes hardware-agnostic. A lesson learned by a robotic arm in one factory can be exported as a .emm file and used by a different model elsewhere.
  • The Fallacy of Self-Judgment: The paper highlights a stark reality: a robot that judges its own success might report 96% completion while only actually solving 35% of tasks. External verification is the only way to bridge this "delusion gap."
  • Matrix Capacity via Widening: Linear matrices usually suffer from "crosstalk" (blurred memories) as they fill up. The authors found that widening the matrix (increasing columns relative to rows) allows it to store significantly more procedures with near-zero error, making it viable for production use.
  • Deterministic Consolidation: Instead of using an LLM to "summarize" experiences, the system uses deterministic rules (e.g., "a sequence must succeed 3 times and cover 60% of cases") to turn raw episodes into a trusted procedure.

Key Results

  • Success Rate Boost: In simulations where the robot's arrival speed was a hidden success factor, a self-judging robot only solved 35% of tasks. Implementing the authors' verified memory increased the success rate to 79% (the theoretical ceiling for the task).
  • Efficiency: Lookups are lightning-fast, taking roughly 16ms on a standard laptop and 40ms on a server node, even with 100,000 episodes stored.
  • Reliability: In revisits to previously solved tasks, the system achieved a 100% success rate on the first attempt once a confirmed record was established.
  • Scalability: A single 1024 x 8192 matrix could successfully distinguish and retrieve 98% of 1,600 synthetic procedures.

Limitations and Future Work

  • The "Cold Start" Problem: Because the system requires external verification, it cannot learn autonomously in environments where no "test" or "human" is available to provide feedback.
  • Sharding Complexity: While the authors discuss "sharding" (splitting memory into many small matrices behind a router), the routing mechanism still loses some accuracy compared to a single, massive exact scan.
  • Simulation vs. Reality: While the UR5e arm was simulated with high fidelity, the authors note that real-world sensor noise and calibration issues remain a challenge for turning "goals" into "motions" across different physical robots.
  • Language Overlap: If goal descriptions are too linguistically similar, the associative matrix can struggle to distinguish between them, requiring more advanced embedding strategies or manual disambiguation.

Summary updated

Abstract

Agents that call tools and robots that execute tasks are challenged by two shortcomings: they repeat what failed before, and they learn from runs that they only believe succeeded. We describe the construction of a memory that addresses both, built for a production agent platform. Every run is recorded as an episode, and only an outcome confirmed by an external check, a test, a tool result or a person counts; the agent's own claim never does. What is learned from episodes is kept in a memory with two parts: an associative matrix, written with the delta rule, which finds the procedure that applies to a situation; and a section of ordered, addressable step records that answers what comes after a given step, where a step is a goal of the agent, not a motion. Because the matrix sums its associations, it keeps no order and cannot return or delete a single one; it serves as the finder, and order, outcome counts and deletion live in the records. We describe how execution outcomes are stored, how episodes are recalled and used for solving tasks, and the file format that stores the memory. In simulation, a robot that judged its own success and stopped when it believed it was done reported $96\%$ of its tasks as done and solved $35\%$; with external verification it solved $79%$. In a store of $100000$ synthetic episodes ($457$MB on disk), the oldest episode was still found first, no episode from another person's memory was returned, and a success vouched for only by the model was never recalled; a lookup took $16$ms on a laptop and $40$ms on a server node (median). Task outcomes were measured up to about $10000$ confirmed records, where the exact records solved every revisit at the first attempt.
BibTeX
@misc{paulhart2026matrix,
  author = {Jürgen Paulhart and Peter Schüller},
  title = {{A Matrix with Steps: An Episodic and Procedural Memory Method for Tool-Using Agents and Robots}},
  year = {2026},
  eprint = {2611.20230},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20230},
}
Figure