AURORA: A Natural Language-Driven Agentic Framework for Understanding, Reasoning, and Orchestrating Reliable Air-Ground Co-Simulation

Keshu Wu, Hao Zhang, Rui Gan, Xiangbo Gao, Xiaopeng Li, Zhengzhong Tu, Yang ZhouarXiv 2026 · arXiv

TL;DR

AURORA is a novel agentic framework that transforms natural-language prompts into reliable, verified air-ground co-simulations by treating scenario generation as a compilation process with strict verification. By introducing an intermediate representation called the Air-Ground Scenario Graph (AGSG), AURORA successfully bridges the gap between high-level LLM planning and low-level simulator execution, enabling automated runtime monitoring and localized failure repair.

What Problem Does This Paper Address?

Testing modern mobility systems—such as autonomous driving, cooperative perception, and the low-altitude UAV economy—requires complex co-simulations where aerial and ground agents interact seamlessly. However, authoring these scenarios manually is incredibly labor-intensive, requiring precise coordination of map geometry, flight paths, communication links, and event triggers.

While generative AI and Large Language Models (LLMs) can generate simulation scripts from natural language, they suffer from a major flaw: silent failures. An LLM-generated script might execute perfectly without throwing a single code error, yet completely fail to realize the user's actual intent (e.g., a tracking drone arriving too late, or a critical V2X warning message being dropped due to occlusion). Current frameworks lack a mechanism to verify whether the requested interactions actually happened during the simulation trace.

Methods

AURORA addresses this by treating scenario generation not as direct code generation, but as compilation with verification.

The core innovation is the Air-Ground Scenario Graph (AGSG), a typed intermediate representation that explicitly maps out the dependencies between agents, missions, events, communication links, and success criteria.

The framework operates in three main phases:

  1. Prompt Understanding & Retrieval: An LLM parses the natural language prompt into a structured, typed specification grounded by a simulator knowledge base.
  2. Scenario Reasoning & Verification: Instead of letting the LLM guess coordinates, deterministic modules handle spatial grounding (ranking road-airspace anchors) and temporal planning. The system then runs a Three-Level Verification pipeline:
  3. L1 (Schema Validity): Structural and symbolic checks.
  4. L2 (Static Feasibility): Geometric checks against actual map topology and obstacle lattices.
  5. L3 (Semantic Consistency): Trace-based runtime verification using quantitative temporal-logic semantics to evaluate properties like visibility and message delivery.
  6. Synchronized Co-Simulation & Closed-Loop Repair: A managed executor synchronizes ground (CARLA) and aerial (AirSim) simulators under a shared logical clock. If a runtime violation is detected, the framework uses the AGSG to localize the failure to specific parameters and applies a Closed-Loop Minimal Repair loop (using a kinematic replay surrogate or guarded LLM edits) to fix the scenario without regenerating it from scratch.

The framework supports five core UAV mission primitives to orchestrate these interactions: UAV mission primitives in AURORA: ascend, hover, track, navigate, and land.

Fig. 1. UAV mission primitives in AURORA: ascend, hover, track, navigate, and land.

Datasets

The authors introduce AURORA-Bench, a dedicated evaluation benchmark designed to measure both execution reliability and semantic realization. It consists of:

  • 200 prompts spanning 12 capability categories (e.g., cooperative perception, communication failure, safety-critical events).
  • 904 annotated requirements defining success conditions.
  • Inclusion of boundary-feasible and strictly infeasible probes to test the framework's rejection capabilities.

Key Insights and Ideas

  • Execution $\neq$ Realization: Simply checking if a simulation script runs to completion is a poor metric for scenario generation. Without runtime verification, standard LLM approaches suffer from silent failures in 34% to 58% of generated scenarios.
  • The Power of Intermediate Representations: By decoupling semantic understanding (LLM) from physical grounding (deterministic algorithms) via the AGSG, the system avoids common LLM pitfalls like spatial hallucination and API mismatch.
  • Differentiable-like Feedback from Logic: Using quantitative temporal logic provides a signed robustness margin (how close a condition was to passing or failing) rather than a binary pass/fail, which directly guides the numerical optimization during the repair phase.

Key Results

The authors evaluated AURORA across five state-of-the-art LLMs (including GPT-4o, GPT-5.5, and Gemini 3.1 Pro):

  • 100% Execution Completion: Introducing the structured AGSG and managed executor completely eliminated runtime execution crashes across all models.
  • Substantial Realization Gains: AURORA boosted the Verified-Pass Rate (scenarios where all semantic constraints were actually met) by 10 to 20 percentage points compared to baselines without runtime repair, achieving overall pass rates between 64% and 84%.
  • Efficient Localized Repair: The closed-loop repair mechanism successfully resolved complex spatial and temporal violations by adjusting only a handful of localized parameters, preventing the need for costly full-scenario re-generation.

Limitations and Future Work

While highly effective, AURORA currently operates under a bounded scenario schema with simplified sensing and communication models, and supports only three primary UAV mission templates (hover, track, navigate). Additionally, because the repair loop is allowed to adjust parameters to make a scenario feasible, a "verified pass" sometimes means the system safely relaxed a user's strict threshold rather than fulfilling the prompt unconditionally.

The authors plan to extend AURORA to support multi-UAV orchestration, richer multi-agent air-ground interactions, closed-loop vision-language-action (VLA) models, and deployment onto real-world robotic testbeds.

Summary updated

Abstract

Air-ground transportation research increasingly relies on co-simulation, yet constructing scenarios remains labor-intensive and difficult to validate. More importantly, a generated scenario may execute successfully while failing to realize the spatial, temporal, communication, or behavioral relationships requested by the user. This paper presents AURORA, a natural-language-driven agentic framework that treats air-ground scenario generation as a process of compilation with verification. Central to AURORA is the Air-Ground Scenario Graph (AGSG), a typed intermediate representation that explicitly connects agents, aerial missions, events, communication links, success conditions, and their cross-domain dependencies. This shared representation enables simulator-grounded parsing, joint road-airspace grounding, temporal planning, pre-execution feasibility checking, trace-based runtime verification, failure localization, and bounded repair within a unified workflow. We further introduce AURORA-Bench to evaluate not only whether generated scenarios execute, but whether they faithfully realize the requested interactions. Experiments across multiple language models show that structured execution substantially improves reliability, while runtime verification exposes silent failures that completion-based evaluation overlooks. Localized repair further resolves many violations without regenerating the entire scenario. The results show that reliable scenario generation requires verifying realized behavior, not merely executable code, and demonstrate the value of explicit intermediate representations for verifiable and repairable language-driven co-simulation.
BibTeX
@misc{wu2026aurora,
  author = {Keshu Wu and Hao Zhang and Rui Gan and Xiangbo Gao and Xiaopeng Li and Zhengzhong Tu and Yang Zhou},
  title = {{AURORA: A Natural Language-Driven Agentic Framework for Understanding, Reasoning, and Orchestrating Reliable Air-Ground Co-Simulation}},
  year = {2026},
  eprint = {2609.19527},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.19527},
}
Figure