REINS: A Self-Evolving Agent Harness for Real-Time Trajectory Planning

Zhihong Cui, Hengyu Liu, Haoran Tang, Shijun Liu, Amir Taherkordi, Tor SkeiearXiv 2026 · arXiv

TL;DR

REINS is a "runtime harness" that allows Large Language Models (LLMs) to perform real-time trajectory planning for autonomous vehicles. By grounding LLM reasoning into executable rules and storing them in a fast, indexed memory, REINS achieves millisecond-level latency and "self-evolves" to handle novel scenarios without ever retraining the underlying model.

What Problem Does This Paper Address?

Large Language Models possess an incredible "semantic prior" for driving—they understand traffic norms, human intent, and safety reasoning in ways classical planners often miss. However, they face a massive deployment hurdle: latency.

Autonomous driving requires decisions within a millisecond-level control budget (typically 50ms or less). LLMs, which take seconds to generate a response, are far too slow to sit directly in the control loop. Previous attempts to bridge this gap either offload the LLM to offline "rule-books" (which makes the system static and unable to adapt to new scenes) or use hybrid designs that lack rigorous safety verification.

The authors of REINS ask: How can we build a system that uses LLM intelligence in real-time while allowing it to learn and "self-evolve" as it encounters new driving environments?

Methods

REINS (Real-time Evolving Indexed Network Scaffold) functions as a runtime scaffold that wraps the LLM. It relies on three critical "couplings" between the LLM and the vehicle's planner:

  1. Grounding: Natural language suggestions (e.g., "keep a safe distance") are translated into physical state variables like Time-to-Collision (TTC) or Distance Headway (DHW). This is achieved through a learned world dynamics model that uses LLM-derived semantic priors to understand variable dependencies.
  2. Verification: Before any LLM-proposed maneuver is executed, the harness "rolls it out" using the dynamics model. If the predicted trajectory violates safety guards or results in a collision, the proposal is rejected.
  3. Consolidation: Verified maneuvers are compiled into executable rules and stored in an indexed graph database (Neo4j).
Figure 1: Three couplings an agent harness must enforce on the LLM planner, illustrated on a car-following scene with a 25-m gap. The LLM utterance “keep a safe distance” is (1) grounded to physical state variables (TTC, THW, DHW), (2) verified by forward rollout over a multi-step horizon, and (3) consolidated as an indexed rule keyed by these variables for instant reuse.

The system operates in a self-evolution loop. When the vehicle encounters a familiar scene, it performs a fast $O(\log n)$ memory lookup to retrieve a rule. If the scene is novel (a "memory miss"), the LLM is invoked asynchronously. Its output is grounded, verified, and—if successful—written back to the memory for future use.

Figure 2: The REINS framework. Phase 1 (offline): learn dynamics ˆG, ˆF and compile rules into initial memory R0. Phase 2 (online): per-scene retrieval and rollout-based verification produce control; on a miss, the LLM proposes a rule that is grounded, verified, and written back to R, closing the self-evolution loop without any LLM weight update.

Datasets

The researchers evaluated REINS across both simulated and real-world environments:

  • Simulation: Four scenarios in highway-env (Highway, Merge, Roundabout, and Intersection) with reactive surrounding vehicles.
  • Real-World: Two drone-captured datasets, highD (German highway) and rounD (German roundabouts), featuring naturalistic human driving behavior.

Key Insights and Ideas

  • Monotonic Decline in LLM Usage: As the "skill memory" matures, the rate of LLM calls drops significantly. In experiments, LLM invocation decreased by 5.5x over 1,000 episodes as the system "learned" the environment.
  • Evolution Without Retraining: Unlike traditional machine learning, REINS "evolves" by accumulating verified symbolic rules rather than updating neural network weights. This makes the learning process transparent and avoids the risks of catastrophic forgetting.
  • Safety as a Filter: By placing a verification step between the LLM and the controller, the system can catch stochastic "hallucinations" or unsafe suggestions before they affect the vehicle's physical behavior.
  • Context-Aware Retrieval: Rather than applying a massive, static rule-book to every situation, REINS uses Cypher queries to retrieve only the 2–3 rules most relevant to the current specific scene variables.

Key Results

REINS outperformed eight major baselines (including classical planners like MPC and other LLM-based methods) across several metrics:

  • Latency: Planning completes in 1.9–4.6 ms, well within real-time requirements and up to 58% faster than other LLM-based planners.
  • Safety: REINS achieved a nearly 50% reduction in collision rates compared to the strongest learned baseline (Plan-R1).
  • Compliance: The system maintained 98%+ compliance with traffic rules.
  • Robustness: The performance gains held up even under 10% observation noise and 5% frame drops.
Performance comparison across simulated and real-world datasets.

Limitations and Future Work

While REINS represents a significant step forward, the authors acknowledge several limitations:

  • The Abstraction Gap: The current evaluation assumes "clean" factored variables (perfect perception). In the real world, the harness would need to handle sensor noise and occlusion.
  • Correlational Dynamics: The world model learns correlations but doesn't necessarily identify true causal relationships, which could lead to conservative behavior in complex interactions.
  • Domain Specificity: While the harness pattern is general, the specific variable schemas and action spaces used in this paper are tailored for driving.

Future work aims to extend REINS to multi-agent coordination and integrate it more deeply with perception stacks to handle uncertainty.

Summary updated

Abstract

Agent harnesses -- runtime scaffolds that wrap an LLM's reasoning loop with tool dispatch, verification, and persistent memory -- have become the standard substrate for deploying LLMs as autonomous agents, as demonstrated by Claude Code and Codex in software engineering. In trajectory planning, however, this paradigm has not taken hold: existing LLM-based trajectory planning methods either keep the LLM in the decision loop, exceeding the millisecond-level control budget, or offload it to offline rules or training-time supervision -- leaving the runtime planner non-evolving. Intermediate hybrids still lack the verification-and-consolidation loop that makes harnesses reliable. We ask: what runtime harness would let an LLM planner meet real-time constraints while self-evolving into a deployable one? We propose REINS, a harness built on three LLM-planner couplings: LLM outputs are (i) grounded to a learned variable-level dynamics model, (ii) verified via calibrated predicates and forward rollout, and (iii) consolidated into an indexed rule memory that serves future scenes in O(log n) without re-invoking the LLM. The harness closes a self-evolution loop: familiar scenes resolve by memory lookup; novel scenes invoke the LLM, whose verified outputs enrich the memory. Without modifying LLM weights, REINS attains 98%+ compliance, nearly 50% collision reduction, and 1.9-4.6 ms latency on simulated and real-world benchmarks at the tactical decision layer, with LLM invocation rate monotonically declining as memory matures. Code: https://github.com/ZhihongCui/REINS
BibTeX
@misc{cui2026reins,
  author = {Zhihong Cui and Hengyu Liu and Haoran Tang and Shijun Liu and Amir Taherkordi and Tor Skeie},
  title = {{REINS: A Self-Evolving Agent Harness for Real-Time Trajectory Planning}},
  year = {2026},
  eprint = {2611.10748},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10748},
}
Figure