TL;DR
PreAct-Nav is a training-free reasoning harness that equips embodied agents with "visual foresight" for complex urban navigation. By using an action-conditioned world model to preview the consequences of movements and a persistent memory to track medium-horizon subgoals, it allows agents to review and revise their plans before ever taking a step.
What Problem Does This Paper Address?
Navigating a city is a "long-horizon" task—you might need to walk blocks, make multiple turns, and identify specific landmarks to reach a destination. Most current AI navigation agents struggle to bridge the gap between these distant goals and the immediate, local decisions they must make (e.g., "should I turn left at this specific mailbox?").
Existing methods often rely on linguistic reasoning over transient observations. While a Vision-Language Model (VLM) can "think" about what might happen, it lacks a visual imagination. It can't "see" that a proposed path is actually blocked by an occlusion or that a turn leads away from the intended route until it has already moved there. This lack of foresight leads to inconsistent decisions and failure on long, complex routes.
Methods
The authors introduce PreAct-Nav, an agentic reasoning harness that wraps around existing, "frozen" navigation policies. It doesn't require retraining the underlying models; instead, it organizes a VLM reasoner and two specialized tools into a four-stage loop:
- Action Preview: The agent checks its Navigation Memory, which maintains a "medium-horizon subgoal" (e.g., "reach the mid-block to see the shops"). If the current observation isn't enough to confirm a move, it queries the Predictive Sandbox.
- Action Review: The Predictive Sandbox uses an Action-Conditioned World Model (AC-WM) to generate a short video rollout of what the agent would see if it executed the proposed action. The VLM reasoner then evaluates this video. If the action contradicts the subgoal, the agent revises the action before execution.
- Action Execute: The agent performs the finalized movement in the real environment.
- Memory Update: The agent compares the actual new observation with what it expected to see. This feedback is used to update the status of the subgoal (Achieved, Active, or Contradicted) and refine the memory for the next step.
Datasets
The framework was evaluated on two major urban navigation benchmarks:
- UrbanNav: A large-scale dataset providing first-person trajectories and language descriptions of destination views.
- CityWalker: A benchmark for learning urban navigation from web-scale human videos.
- Real-World Demonstration: The authors also deployed the system on a physical wheeled robot in an outdoor plaza to demonstrate closed-loop reasoning in the wild.
Key Insights and Ideas
- Visual Foresight > Linguistic Guessing: The "aha moment" here is that a VLM is much more effective at navigation when it can interpret a predicted video rather than just reasoning about future landmarks in text. The visual cues in the "sandbox" rollouts provide concrete evidence of progress or failure.
- Subgoals as Anchors: By maintaining persistent medium-horizon subgoals in memory, the agent avoids the "myopia" of local decision-making. It ensures that every small step serves a larger, intermediate milestone.
- Closing the Loop with Feedback: The system doesn't just predict; it verifies. By comparing real outcomes to predicted ones, the agent can "realize" when its assumptions were wrong (e.g., "I thought I could pass these pedestrians, but they are still blocking the path") and update its strategy immediately.
Key Results
PreAct-Nav demonstrated significant performance gains across multiple navigation policies (like NoMaD and OmniVLA) and VLM reasoners (Qwen and GPT-5.6 Sol):
- UrbanNav: Achieved relative improvements of 6.71% in Success Rate (SR) and 4.19% in Success weighted by Path Length (SPL) over the base policy.
- Complexity Scaling: The benefits were most pronounced on the hardest tasks. On routes longer than 20 meters, the SR gain reached up to 5.57 percentage points. Similarly, gains were higher on routes involving more cumulative turning.
- CityWalker: The method outperformed the strongest baseline (NavGPT) by 5.84 SR points.
Limitations and Future Work
The primary bottleneck for PreAct-Nav is computational cost. Generating high-quality video rollouts is slow; the authors note that video generation takes roughly 11.37 seconds per rollout, which dominates the inference time. This makes the current framework unsuitable for high-speed, real-time robotics.
Additionally, the system is limited by the fidelity of the world model. If the world model "hallucinates" a clear path that is actually blocked, the VLM reasoner might be misled. Future work will likely focus on faster, more reliable world models and extending this "reasoning-before-action" loop to more dynamic, time-sensitive environments.
Summary updated