TL;DR
When large language model (LLM) weights are frozen behind APIs or deployment constraints, how can an agent adapt to entirely new environments? This paper introduces Workspace Optimization, a framework that treats an agent's external substrate (code, hypotheses, and logs) as the trainable state, updating it via a structured "commit-then-retrodict" loop. Implemented as a multi-agent system called DREAMTEAM, this approach sets a new state-of-the-art on the interactive ARC-AGI-3 benchmark while using 31% fewer environment actions than previous methods.
What Problem Does This Paper Address?
Traditional machine learning relies on weight-space adaptation: data produces a loss, which generates a gradient, updating the model's internal parameters. However, many real-world agent deployments cannot use this recipe. Fine-tuning is too slow for online, within-episode adaptation, and weight access is often blocked entirely when models sit behind commercial APIs.
This leaves a fundamental question: What is the trainable state when weights are off the table?
The authors tackle this challenge in the context of hard, multi-turn, unfamiliar environments—specifically the ARC-AGI-3 benchmark—where an agent is dropped into a game with unknown rules, mechanics, and goals. To succeed, the agent must learn purely through interaction, requiring a structured way to accumulate knowledge and adjust behavior on the fly without changing a single weight vector.
Methods
The core proposal is Workspace Optimization, a paradigm that mirrors the computational graph of traditional training but shifts the substrate from weights to an external, editable workspace ($W_t$):
- Artifacts (Python code, structured logs) replace parameters.
- Evidence (interaction traces) replaces training data.
- Counterexamples (contradicted predictions) replace losses.
- Textual feedback and code patches replace gradients.
The authors instantiate this framework in DREAMTEAM, a multi-agent orchestrator inspired by model-based reinforcement learning architectures like DreamerV3. Instead of neural networks, DREAMTEAM uses six specialized LLM-backed roles that interact through a shared file system:
- Observer: Parses raw environment screens into a structured state ($z_t$) and verifies it by rendering it back to pixels.
- Simulator: Maintains the hidden state history ($h_t$) and predicts the next step ($\hat{z}_{t+1}$). If a prediction fails, it patches the underlying Python dynamics rules.
- Inductive Explorer (IE): Generates and maintains a library of reusable strategies and sub-goals.
- Transductive Explorer (TE): Proposes information-seeking "probes" to explore unmapped or uncertain mechanics.
- Critic: Acts as the failure router, auditing proposals against historical evidence and assigning blame to specific roles.
- Team Leader (TL): The central arbitrator that decides whether to execute a plan, run an exploratory probe, or trigger a repair.
To prevent the agent from "overfitting" its code patches to the most recent failure, the harness maintains a Regression Set ($R_t$) of past transitions. Every proposed code modification is replayed against this history to catch regressions before an action is committed to the environment.
Datasets
The system is evaluated on the ARC-AGI-3 public set (25 games). This benchmark is explicitly designed to test abstract reasoning and sample efficiency in interactive environments. The rules, transition dynamics, and rewards are completely hidden from the agent, mimicking the challenge of a human figuring out an unfamiliar video game without instructions.
Key Insights and Ideas
- Credit Assignment via Addressing: In weight-space training, backpropagation uses the chain rule to distribute a scalar loss across millions of weights. In Workspace Optimization, credit assignment is simplified into an addressing decision. Because the workspace is partitioned into typed, named slots (e.g.,
observable.py,dynamics.py), any specific prediction failure directly points to its structural owner. A parse mismatch goes to the Observer; a physics misprediction goes to the Simulator. - Code as an Inspectable World Model: Instead of learning a latent neural transition model, DREAMTEAM builds its world model out of executable Python code. This makes the model entirely inspectable, falsifiable, and capable of running exact, fast rollouts for look-ahead planning.
- The Epistemic Control Loop: Actions in DREAMTEAM serve a dual purpose. Early in a game, when the world model is immature, the system prioritizes exploration (probes) to reduce epistemic uncertainty. As the Simulator's prediction accuracy stabilizes, control automatically shifts toward model-based planning and highly efficient execution.
Key Results
DREAMTEAM was evaluated using the official ARC-AGI-3 scoring protocol, measuring Relative Human Action Efficiency (RHAE):
- Performance Leap: Averaged over two independent runs, DREAMTEAM achieved 38.4% RHAE, surpassing the previous state-of-the-art protocol-matched baseline (Symbolica's Agentica SDK) at 36.08%.
- Superior Action Efficiency: DREAMTEAM achieved its higher score while using 31% fewer environment actions per game (a mean of 444 actions vs. the baseline's 643).
- Planning Advantage: On successfully solved levels, DREAMTEAM averaged 54.1 actions compared to Symbolica's 100.1 actions—a 46% reduction—highlighting the massive advantage of planning over an explicit, self-corrected world model.
Limitations and Future Work
- The Late-Game Human Gap: While DREAMTEAM beats budget-matched humans in the first few hours of a task due to systematic probing, humans pull ahead in longer sessions (past ~5 hours). The authors attribute this to humans' superior cross-level transfer; humans reuse abstract mechanics cost-free, whereas DREAMTEAM often rebuilds parts of its workspace from scratch at major transitions.
- API Dependency and Latency: The architecture relies heavily on commercial, hosted LLM APIs (e.g., Claude 4.6, GPT-5.5). Endpoint timeouts, rate limits, and latency fluctuations act as external bottlenecks to the agent's real-time performance.
- Open Research Questions: The authors note that treating the workspace as the primary object of training opens up a brand-new research agenda parallel to traditional ML: Which code abstractions should be exposed? How should counterexamples be optimally retained? What repair operators will generalize best across completely different environments?
Summary updated