TL;DR
Kintsugi is a novel white-box policy-learning framework that treats embodied agent improvement as software maintenance—specifically, debugging and repairing an executable, typed Knowledge Base (KB). By shifting Large Language Models (LLMs) out of the real-time inference loop and using them strictly offline to patch localized policy errors, Kintsugi achieves state-of-the-art performance on long-horizon tasks with zero test-time LLM calls.
What Problem Does This Paper Address?
When a robot or virtual agent repeatedly fails a task—like missing a precondition or opening the wrong container—how do we fix it?
In modern embodied AI, task knowledge is typically locked away in opaque neural weights (end-to-end learning) or trapped in brittle, expensive, prompt-bound memories (LLM-as-a-policy). The former lacks inspectability and is prone to catastrophic forgetting during fine-tuning, while the latter is slow, hallucinatory, and costly at inference time.
The authors identify a critical gap: the absence of a structured, policy-maintenance object that can translate rollout failures into durable, verifiable, and localized policy corrections.
Methods
Kintsugi introduces a "policy-as-editable-knowledge" paradigm consisting of four main components:
- Typed Executable Knowledge Base (KB): The policy is represented as a white-box artifact split into distinct, typed layers ranging from low-level grounding ($L0$) and predicates ($L1$) to high-level policy schemas ($L4$) and goal specifications ($L7$).
- Agentic Editing Loop: Between rollouts, a restricted LLM editor acts as a debugger. It analyzes failure trajectories, localizes the error to a specific KB layer, and proposes a precise, typed edit (a
KBDiff). - Deterministic Verification Gate: The proposed edit is never trusted blindly. A deterministic applier type-checks the payload, and a verifier runs it through focused validation and protected-regression checks to ensure the patch fixes the target failure without breaking previously mastered tasks.
- Zero-LLM Execution: At inference time, the LLM is completely sidelined. A deterministic symbolic executor runs the frozen, verified KB directly, mapping environmental states to actions with absolute traceability.
Datasets
The framework was evaluated across two distinct paradigms:
- Text-Based Agents: ALFWorld (interactive household tasks), WebShop (e-commerce shopping environments), and TextCraft (Minecraft-inspired crafting recipes).
- Object-Centric Manipulation (Robotics): MetaWorld (multi-task robotic arm manipulation), Predicators (symbolic-induction planning), and RoboSuite (continuous control simulation used as a boundary stress test).
Key Insights and Ideas
- LLMs as Debuggers, Not Actors: Instead of forcing an LLM to reason about what action to take at every single environment step, Kintsugi uses the LLM as an offline code-repair tool. This maximizes the LLM's reasoning strengths while eliminating its test-time latency and unpredictability.
- Policy Improvement as Monotonic Growth: Because every accepted edit must pass a strict regression gate, the agent's capabilities grow monotonically. It is mathematically protected against backsliding on tasks it already knows how to solve.
- Traceability and Auditing: Every action taken by the executor can be traced back to the exact KB entry, predicate, or rule that triggered it. If something goes wrong, there is a clear audit trail showing which edit introduced the behavior and what evidence justified it.
Key Results
Kintsugi matched or outperformed state-of-the-art language agents that rely on active, test-time LLM reasoning:
- Ceiling Performance: Achieved a 100% success rate on both ALFWorld and TextCraft, outperforming popular baselines like ReAct, Reflexion, and Dual Memory.
- WebShop Gains: Reached a success rate of 52% (Score: 0.7392), surpassing all prompt-based and memory-augmented baselines.
- Robotic Competence: In MetaWorld (MT-10 and MT-50), Kintsugi achieved 99.2% and 98.32% success rates respectively, proving that the executable KB framework seamlessly translates to continuous, object-centric physical simulations.
Limitations and Future Work
While Kintsugi excels at task-level logic, it relies on a pre-existing, reliable perception and low-level control substrate. It assumes the environment can expose structured object states (e.g., bounding boxes, open/closed states) to the $S0/L0$ layers. If a visual tracker fails or an object is misdetected, Kintsugi currently cannot fix that perceptual error on its own.
The authors note that the immediate next step for future work is replacing the oracle/simulator state interface with a vision-derived state interface, turning raw visual inputs into the object slots and tracks that Kintsugi's symbolic executor needs.
Summary updated