System-One RSI: Building Muscle Memory for Interactive Agents through Guided Practice

Qingxu Fu, Xuanfa Jin, Ziqian Chen, Zhaoyang Liu, Bolin DingarXiv 2026 · arXiv

TL;DR

System-One RSI (Recursive Self-Improvement) enables AI agents to perform high-speed, real-time tasks by distilling slow, deliberate reasoning into fast "muscle memory." By coupling an online execution loop with an offline "coaching" loop, the system learns to make split-second visual judgments in complex games like Black Myth: Wukong using only a handful of real-world interaction episodes.

What Problem Does This Paper Address?

Large Language Models (LLMs) are incredibly smart at reasoning and planning, but they are notoriously slow. In a high-stakes action game, a "dodge window" might only last 200 milliseconds. If an agent spends two seconds "thinking" and generating a reasoning trace, the character is already dead.

Conversely, traditional Reinforcement Learning (RL) is fast during execution but typically requires thousands of trials in a simulated environment. Many real-world applications—and modern games with persistent states or anti-cheat measures—cannot be easily sandboxed or run in parallel at scale. The authors tackle the "missing middle": how to give an agent the sophisticated task-understanding of a reasoning model while maintaining the sub-second reaction speeds required for real-time interaction, all while working within a very limited "data budget" of real-world experience.

Methods

The researchers introduce a dual-loop architecture inspired by human motor learning:

  1. The Inner Loop (System-One / "The Athlete"): This is a high-frequency Observe, Orient, Decide, Act (OODA) cycle. It uses a 27B parameter "decision model" (specifically a JEV-style model). Unlike standard LLMs, this model doesn't generate text; it performs a single forward pass to return structured judgments (e.g., "Is an attack incoming?"). These judgments are fed into a decision tree that triggers specific keyboard/mouse actions. This keeps latency low (~200ms).
  2. The Outer Loop (RSI / "The Coach"): This loop runs offline. A team of reasoning agents (LLMs) acts as a coach. They review the "game tape" (interaction logs), identify failures, and provide supervision. Crucially, they label specific frames to teach the decision model fine-grained distinctions—like the subtle difference between a boss winding up an attack and the actual moment you need to dodge.
  3. Capability Consolidation: The system uses LoRA (Low-Rank Adaptation) to fine-tune the decision model based on the coach's feedback. This "bakes" the reasoning into the model's weights, effectively building "external muscle memory."
Figure 2: Guided practice as a model for learning external muscle memory.

Datasets

The paper avoids massive pre-existing datasets, focusing instead on "live" adaptation in three dynamic visual environments:

  • Snake: Used as a baseline for visual navigation and obstacle avoidance.
  • Zenless Zone Zero (ZZZ): A fast-paced action RPG used to test attack-warning recognition and character switching.
  • Black Myth: Wukong: A high-fidelity action game used to evaluate precise dodge timing and offensive combat strategies against difficult bosses.

The "dataset" for each task is generated on-the-fly, typically consisting of only 5 to 20 episodes per adaptation round.

Key Insights and Ideas

  • The "Coach" Analogy: Humans don't learn to ride a bike by reading physics papers; they learn through practice and a coach who points out specific errors. RSI mimics this by having an LLM identify why a failure happened, rather than just relying on a generic reward signal.
  • Weight Updates vs. Prompting: For tasks like dodging, visual nuances are too subtle to describe in a prompt. As shown in Figure 1, the difference between a "pre-attack" posture and a "dodge window" is minute. By updating the model's parameters (LoRA), the agent learns to "feel" the timing rather than "reason" about it.
Successive observations of an attack in Black Myth: Wukong showing the subtle dodge window.
  • Separation of Concerns: By separating the judgment (the neural network) from the action logic (the decision tree), the system becomes more interpretable. You can update the "rules" of the game without retraining the "eyes" of the agent, or vice versa.

Key Results

The framework demonstrated significant performance leaps across all tested environments:

  • Snake: The mean score jumped from 1.30 to 7.35 after adaptation.
  • Zenless Zone Zero: The success rate against bosses rose from 25% to 100%, with a median completion time reduction of about 8%.
  • Black Myth: Wukong (Evasion): The agent's mean survival time increased from 23.6s to 260.5s (an 11x improvement).
  • Black Myth: Wukong (Offense): A base model with zero wins was transformed into an offensive powerhouse with a 66.7% win rate after recursive improvement.

Limitations and Future Work

  • Dependence on Priors: The system requires a reasoning model that already has some "common sense" or task knowledge to act as an effective coach. If the initial "coach" can't identify what's important, the "athlete" won't learn.
  • Hardware Constraints: While the decision path is fast, running a 27B parameter model at 5Hz still requires significant GPU resources.
  • Generalization: The "muscle memory" learned is highly task-specific. Moving an agent from one boss to another with completely different animations would likely require a new round of RSI.
  • Supervision Quality: The authors note that a recursive loop isn't guaranteed to improve; if the reasoning model provides incorrect labels, it can lead to "hallucinated" muscle memory that degrades performance.

Summary updated

Abstract

Reasoning models provide strong multimodal task understanding but struggle with sub-second control because deliberation increases decision latency. Fast decision models can meet the response window, yet prompting and few-shot examples do not readily provide the fine-grained experience required by a new task. Inspired by coached motor learning, this paper presents System-One Recursive Self-Improvement (System-One RSI), which couples two loops. The high-frequency online inner loop is an Observe, Orient, Decide, Act (OODA) cycle: a decision model maps each visual observation and task question to structured judgments, and a decision tree converts them into timed actions without invoking a reasoning model. The offline outer loop is a recursive improvement cycle carried out by a team of reasoning agents. The team revises the decision-model prompt and decision tree, captures interaction records, derives supervision from those records, and trains or deploys updated model parameters. The improved system then returns to the environment and generates the experience for the next round, consolidating learned distinctions as external muscle memory. The framework operates with a small number of real episodes, without expert demonstrations, environment modification, sandboxing, or parallel simulation. Experiments in Snake, Zenless Zone Zero, and Black Myth: Wukong use a 27B decision model. Mean Snake score increases from 1.30 to 7.35. In an evaluated scenario of Zenless Zone Zero, median completion time decreases by 8.4%. In an evaluated combat scenario of Black Myth: Wukong, the evasive policy achieves 11 times the mean survival time of a stationary reference, while the offensive policy win rate rises from 0% to 66.7% across the evaluated configurations.
BibTeX
@misc{fu2026system,
  author = {Qingxu Fu and Xuanfa Jin and Ziqian Chen and Zhaoyang Liu and Bolin Ding},
  title = {{System-One RSI: Building Muscle Memory for Interactive Agents through Guided Practice}},
  year = {2026},
  eprint = {2611.26007},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.26007},
}
Figure