TL;DR
SARBLOH is a new autonomous agent architecture that achieves state-of-the-art performance on the interactive ARC-AGI-3 benchmark by treating spatial perception as the primary bottleneck for AI reasoning. By replacing raw integer grid inputs with a deterministic perception engine that extracts geometric invariants and testable hypotheses, SARBLOH achieved a 16.53 score on the official hidden Kaggle benchmark—a 13.7x improvement over the previous milestone winner.
What Problem Does This Paper Address?
The ARC-AGI-3 benchmark challenges AI agents to solve unfamiliar, uninstructed visual games. Unlike the original ARC, this version is interactive: agents must explore the environment to deduce rules. A critical hurdle is the Relative Human Action Efficiency (RHAE) metric, which penalizes unnecessary exploratory actions quadratically.
The author identifies a fundamental failure mode called Perceptual Misinduction. Most current models receive the environment as a raw matrix of integers (e.g., a 64x64 grid). Serializing this grid into text tokens destroys the "spatial topology"—the model no longer "sees" that two cells are adjacent. Consequently, frontier models squander their action budgets just trying to rediscover basic boundaries and symmetries that a human would perceive instantly.
Methods
The core of the paper is SARBLOH, an open-weight agent (based on Qwen-3.8-Flash-Next) that shifts the burden of spatial reasoning from the neural network to a deterministic, host-side perception engine.
The Measured–Guessed Rule
SARBLOH operates on a strict dichotomy:
- Measured Facts: Geometric invariants like reachability, enclosures, and symmetries ($D_4 \times \mathbb{Z}^+$) are computed deterministically in $O(HW)$ time. These are presented to the model as absolute truths.
- Guessed Roles: Functional affordances (e.g., "Is this object the player?") are treated as probabilistic hypotheses. Each guess is paired with a specific one-action verification test to confirm or refute it immediately.
The Perception Engine
Instead of raw grids, the agent receives a "Fused Briefing" consisting of:
- Object Inventory: A tracked list of monochromatic regions that maintains identity across frames.
- Annotated Canvas: A visual blueprint that highlights structures like floors, walls, and HUD elements.
- Geometric Briefing: A text summary of spatial relations (e.g., "obj 6 is reachable in 6 steps").
Code-as-Action Runtime
SARBLOH interacts via an embedded IPython kernel. Instead of emitting low-level motor primitives (like "Move Up"), it synthesizes algorithmic hypotheses in Python. It uses a speculative await play() primitive that executes multi-step trajectories but halts immediately if the environment's state deviates from the agent's internal model.
Datasets
- ARC-AGI-3 Public Benchmark: 25 interactive environments used for comparative analysis against frontier models like GPT-6.1 and Claude Opus 5.
- Official Hidden Kaggle Evaluation Suite: A set of undisclosed games used to verify the final competition score under strict 9-hour and single-GPU constraints.
- Held-out Games: 23 official games used to test the cross-distribution generalization of the perception engine.
Key Insights and Ideas
- Perception > Planning: Ablation studies (Figure 2) reveal that over 90% of efficiency gains come from structuring the perceptual representation rather than increasing the depth of the model's planning.
- Quadratic Penalties Favor Internal "Thinking": Because the benchmark penalizes physical actions quadratically but does not penalize internal computation time, it is always optimal to perform extensive host-side geometric parsing and speculative planning before committing to a move.
- Token Efficiency: By abstracting the grid into objects and relations, SARBLOH reduces the prompt token footprint by 89.4% (from ~3,500 to ~370 tokens per frame), which significantly increases the capacity of the GPU's prefix cache.
Key Results
- Benchmark Dominance: SARBLOH achieved an official hidden Kaggle score of 16.53, advancing 13.7x beyond the first Milestone winner.
- Surpassing Frontier Models: On public environments, SARBLOH (an open-weight model) achieved a peak mean efficiency of 72.5%, outperforming proprietary API-based models like GPT-6.1 (54.8%) and Claude Opus 5 (37.8%) on 21 of 25 games.
- Human-Level Efficiency: In "Run B" (full briefing active), the agent recorded a median action ratio of 0.68x relative to human baselines, meaning it often solved levels with fewer actions than a median human.
Limitations and Future Work
- Stateless Hypotheses: The current perception engine evaluates hypotheses independently at each frame. The authors suggest moving toward a Closed-Loop Belief Updating system that maintains a Bayesian belief distribution across the entire episode.
- General-Purpose Weights: The agent currently relies on general-purpose model weights. Offline Trajectory Fine-Tuning (LoRA) on successful, high-efficiency execution traces could further optimize code generation.
- Reinforcement Learning: Exploring sample-efficient RL algorithms like GRPO to optimize planning depth within the fixed computational envelope of the competition.
Summary updated