SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning

Yucheng Deng, Pingrui Lai, Xinhai Li, Chenjia Bai, Xiaoheng Deng, Chengnuo Sun, Xuelong Li, Hua YangarXiv 2026 · arXiv

TL;DR

SpaceVLN is a training-free, zero-shot navigation agent that masters complex 3D environments by combining a hierarchical online spatial memory with task-guided spatial reasoning. By breaking down long-form language instructions into verifiable "space-landmark" stages, it achieves state-of-the-art zero-shot performance across multiple benchmarks and successfully transfers to physical robotic platforms.

What Problem Does This Paper Address?

Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires an embodied agent to follow natural language instructions (e.g., "Walk past the bed, turn left at the sofa, and stop by the French doors") inside unfamiliar 3D spaces.

While large Vision-Language Models (VLMs) have enabled "zero-shot" navigation without task-specific training, existing agents suffer from a major flaw: they rely heavily on local visual cues and linear, text-based history. They lack a true, structured understanding of the physical space they are traversing. Consequently, agents easily get disoriented in multi-room layouts. While pre-built maps can alleviate this, they require costly, impractical pre-exploration phases. SpaceVLN bridges this gap by building a structured, relational spatial memory entirely on-the-fly.

Figure 1: Comparison between prior navigation strategies and SpaceVLN. (a) Prior strategies rely on task-specific training, direct VLM action prediction, or pre-built maps, limiting generaliza- tion and weakening spatial progress grounding. (b) SpaceVLN is a zero-shot, training-free navi- gation agent. It leverages online Spatial Cognitive Memory and Task-Guided Spatial Reasoning to strengthen the agent’s spatial cognition and reasoning ability, thereby improving navigation success.

Methods

SpaceVLN introduces a stagewise closed-loop navigation framework that decouples high-level spatial planning from low-level execution control.

Figure 2: Framework of SpaceVLN. SpaceVLN realizes navigation through a stagewise closed- loop framework, augmented by Spatial Cognitive Memory and Task-Guided Spatial Reasoning. The planner infers progress from 12-direction look-around and spatial memory, then generates a verifi- able space–landmark stage as an executable subtask. The executor follows this subtask using FPV views and landmark memory, while closed-loop updates memory for verification and replanning.

The architecture consists of three core pillars:

  1. Stagewise Closed-Loop Framework: Instead of predicting low-level actions directly from a VLM, a high-level Planner analyzes the environment and generates a verifiable subtask called a "space-landmark stage" (e.g., moving from the current room to a specific landmark). An efficient, low-level Executor then handles high-frequency control (moving forward, turning, avoiding obstacles) to reach that landmark. After each stage, the agent verifies its progress and replans if necessary.
  2. Hierarchical Spatial Cognitive Memory: As the agent moves, it builds a dual-layer memory online:
  3. Hierarchical Spatial Memory: Dynamically abstracts explored regions into an agent-centered "Spatial Waypoint" graph, mapping out connectivity across rooms and floors without needing dense 3D reconstruction.
  4. Local Landmark Memory: Maintains a localized pool of subtask-grounded landmarks using open-vocabulary object detectors (GroundingDINO and RepViT-SAM) to track target objects precisely during execution.
  5. Spatial-CoT (Chain-of-Thought): A schema-guided reasoning mechanism that forces the VLM to explicitly output structured fields covering observation analysis, progress estimation, and next-stage planning. This prevents the model from hallucinating and ensures its spatial reasoning aligns with its physical memory.
Figure 3: Spatial Cognitive Memory. (a) Hierarchical Spatial Memory abstracts views and tra- versed paths into a Spatial Waypoint graph and executed Spatial Waypoint chain, then converts them into enhanced egocentric spatial representation for planner. (b) Local Landmark Memory maintains a subtask-specific landmark pool, providing top-K cues for execution and next-stage planning.

Datasets

The authors evaluated SpaceVLN across four distinct simulation benchmarks and a physical robot deployment:

  • R2R-CE & RxR-CE: Standard continuous-environment benchmarks for instruction-following navigation.
  • GN-Bench: A benchmark featuring highly complex, multi-turn spatial instructions.
  • HM3D-OVON: An open-vocabulary Object-Goal navigation dataset used to test the agent's ability to find specific objects without explicit path instructions.
  • Real-Robot Deployment: 50 custom indoor navigation episodes conducted on a physical TX-Q1 differential-drive mobile base equipped with an Intel RealSense D435i camera and a Livox LiDAR.

Key Insights and Ideas

  • Decoupling Planning and Execution is Key: Handing high-frequency control over to a lightweight executor while reserving the heavy VLM reasoning for discrete "milestones" dramatically improves both efficiency and accuracy.
  • Relational Memory Beats Text History: Linear text histories fail to capture loops, detours, or multi-floor structures. Abstracting the environment into a topological waypoint graph allows the VLM planner to accurately localize its progress.
  • Verifiable Stages Enable Self-Correction: By defining explicit success criteria for each subtask (e.g., arriving within a specific distance of a validated landmark), the agent can detect when it is lost or blocked and actively replan its route.

Key Results

SpaceVLN achieved state-of-the-art zero-shot performance across all evaluated benchmarks, frequently outperforming models that relied on supervised training or pre-exploration maps:

  • R2R-CE: Achieved a 53.3% Success Rate (SR), outperforming the previous zero-shot state-of-the-art (GTA) by a relative gain of 9.2%.
  • RxR-CE: Reached a 48.9% SR, marking a 5.8% relative improvement over prior methods.
  • GN-Bench & HM3D-OVON: Secured top zero-shot performance with 39.3% SR and 51.6% SR, respectively.
  • Real-Robot Transfer: Maintained strong performance in physical environments, achieving an overall 48% success rate and proving highly resilient against real-world sensing noise and motion drift.

Limitations and Future Work

Despite its strong performance, the authors note several limitations:

  • Perception Dependency: The system remains sensitive to open-vocabulary grounding errors and noisy object detection labels, which can corrupt the landmark memory.
  • Geometric Planning Gaps: Because the agent lacks a long-horizon geometric pathfinder, it can get stuck in highly complex layouts requiring large detours or navigating around heavily occluded obstacles.
  • Inference Latency: Repeatedly querying large foundation models at every planning boundary introduces latency. The authors suggest that future work should focus on developing more compact spatial memories and lighter control loops to make deployment even more practical.

Summary updated

Abstract

Vision-and-Language Navigation in continuous environments requires agents to understand the spatial structure of previously unseen environments in order to follow language instructions. Although foundation models have opened a promising path toward zero-shot navigation without task-specific policy training, many navigators still rely on local visual cues and linear history-based reasoning, overlooking the spatial nature of navigation across explored regions, traversed paths, landmarks, and their spatial relations. In this paper, we propose SpaceVLN, a navigation agent built around Spatial Cognitive Memory and Task-Guided Spatial Reasoning. Specifically, SpaceVLN introduces an efficient stagewise closed-loop framework where planning and execution are organized around verifiable space--landmark stages. During navigation, the agent progressively abstracts explored regions into Spatial Waypoints and dynamically maintains subtask-grounded landmark evidence, forming a hierarchical Spatial Cognitive Memory for progress localization and spatial-relation understanding. Built on this memory, Spatial-CoT integrates task-progress reasoning with spatial perception, analysis, and prediction, enabling Task-Guided Spatial Reasoning for embodied navigation. The unified stage interface enables SpaceVLN to address both Vision-and-Language Navigation and Object-Goal Navigation under a unified zero-shot setting, without task-specific policy training. Across R2R-CE, RxR-CE, GN-Bench, and HM3D-OVON, SpaceVLN achieves state-of-the-art zero-shot performance, and real-robot deployment further validates its applicability. These results highlight Spatial Cognitive Memory and Task-Guided Spatial Reasoning as a practical foundation for stronger embodied navigation agents.
BibTeX
@misc{deng2026spacevln,
  author = {Yucheng Deng and Pingrui Lai and Xinhai Li and Chenjia Bai and Xiaoheng Deng and Chengnuo Sun and Xuelong Li and Hua Yang},
  title = {{SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning}},
  year = {2026},
  eprint = {2606.08992},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.08992},
}

Loading related content...

Figure