VLAN: Vision-Language Accessible Navigation

Jie Hu, Yu Zheng, Wenzhi Wu, Weixing Feng, Cunlei Cheng, Jiaming ZhangarXiv 2026 · arXiv

TL;DR

VLAN (Vision-Language Accessible Navigation) is a new large-scale benchmark that shifts the focus of embodied AI from "can this agent reach the goal?" to "is this path safe and accessible for this specific user?" By providing 20,000 trajectories across five distinct embodiments—ranging from drones to wheelchair users—the researchers demonstrate that even state-of-the-art vision-language models (VLMs) struggle to adapt their safety reasoning and path planning to the unique physical constraints of different users.

What Problem Does This Paper Address?

In traditional embodied navigation research, traversability is often treated as a scene-intrinsic property: a path is either "walkable" or "blocked." However, in the real world, accessibility is user-conditioned. A low barrier might be a simple step for a humanoid robot, a high-risk obstacle for a blind (BVI) user, and a total dead-end for someone in a wheelchair. Conversely, overhanging tree branches are critical hazards for drones but irrelevant to quadruped robots.

Existing benchmarks typically evaluate a single, generic agent. This "one-size-fits-all" approach ignores the diverse physical, perceptual, and operational constraints of different human and robotic users. The authors argue that for AI to be truly inclusive and useful in complex urban environments, it must reason about the environment through the "eyes" and physical capabilities of its specific target embodiment.

The core concept of VLAN: accessibility depends on the user's mobility and perception.

Methods

The authors developed a sophisticated real-to-interactive simulation pipeline to create a diverse and controllable dataset:

  1. Scene Reconstruction: They collected egocentric videos of real-world urban scenes and reconstructed them into 3D Gaussian Splats.
  2. Interactive Editing: These reconstructions were imported into the Unity game engine. Unlike static videos, this allowed the researchers to perform "counterfactual" editing—changing traffic light states, adding or moving obstacles, and altering environmental conditions (e.g., making the ground wet or snowy).
  3. User-Conditioned Annotation: They manually annotated accessibility for five embodiments: humanoid robots, quadruped robots, drones, BVI users, and wheelchair users. Obstacles were categorized into four levels: Passable, Bypass, HighRisk, and Blocked.
  4. Task Formulation: The benchmark evaluates two distinct task families:
  5. Action Prediction: For drones and quadrupeds, models must output low-level control signals ($\Delta x, \Delta z, \Delta \theta$).
  6. Language-Guided Navigation: For humanoid, BVI, and wheelchair users, models must generate high-level, user-specific natural language instructions (e.g., using clock directions and step counts for BVI users).
The VLAN scene reconstruction and editing pipeline.

Datasets

The VLAN Dataset is a significant contribution in its own right:

  • 1,000 unique scenes reconstructed from real-world data.
  • 20,000 navigation trajectories and 160,000 keyframes.
  • Diverse Conditions: The dataset spans daytime, nighttime, dusk, and low-vision settings, as well as various weather conditions (sunny, rainy, snowy, wet ground).
  • Semantic Variety: Includes urban, suburban, rural, and indoor environments with varying path geometries (wide, narrow, open, constrained).
Visualizing the diversity of scenes in the VLAN dataset across lighting, type, geometry, and weather.

Key Insights and Ideas

  • The "Embodiment Gap": A model's success on one type of agent does not guarantee success on another. The researchers found that general-purpose VLMs often fail to consistently adapt their obstacle relevance and safety criteria across different user profiles.
  • Perception $\neq$ Planning: A fascinating "aha moment" came from analyzing failures. Many models correctly identified the obstacle in language ($P^+$) but still failed to generate a safe navigation action ($A^-$). This suggests that the bottleneck isn't just visual recognition, but the spatial grounding of that recognition into actionable, safety-aware plans.
  • Conservative vs. Capable: Smaller models (like Qwen2.5-VL-3B) often showed high "collision-free" rates but very low success rates. They effectively chose to "stay safe by doing nothing," highlighting the need for metrics that balance safety with goal-reaching.
  • Instruction Style Matters: For BVI users, instructions need to be step-based and risk-explicit. For wheelchairs, the focus shifts to clearance and surface stability. The paper shows that these aren't just stylistic choices but functional requirements for different users.

Key Results

The researchers benchmarked seven modern open-weight VLMs (and two closed-source models for reference).

  • Top Performer: InternVL3.5-8B emerged as the most balanced model, achieving the highest Accessible Success Rate (AccSR)—a harmonic mean of success across all tasks and users.
  • Action Prediction: For drones, InternVL3.5-8B led with a success rate ($SR_{act}$) of 0.642. For quadrupeds, Qwen3-VL-8B-Instruct performed best at 0.474.
  • Language Guidance: Llama-3.2-11B-Vision-Instruct excelled at humanoid guidance, while Molmo2-8B was strongest for wheelchair-specific instructions.
  • Closed-Source Reference: Gemini 3.6 showed impressive performance on drone and BVI tasks (reaching 0.900 success on a drone subset) but, surprisingly, was outperformed by open-weight models in humanoid and wheelchair settings. This reinforces the idea that "general intelligence" does not automatically translate to "universal accessibility."

Limitations and Future Work

  • Embodiment Scope: The current work does not yet cover users with hearing or cognitive impairments.
  • Drone Simplification: The drone task is currently restricted to a fixed-altitude 2D plane to keep it comparable with ground agents. Real-world drone navigation would require 3D reasoning.
  • Sim-to-Real: While a small 30-trajectory real-world test showed promising correlation, more extensive physical deployment is needed to fully validate the benchmark's real-world utility.
  • Future Directions: The authors suggest that future work should incorporate richer multimodal inputs (like audio or haptics) and focus on developing more personalized navigation models that can learn an individual user's specific preferences and capabilities.

For more details, check out the full paper or explore the code and data at https://github.com/InSAI-Lab/VLAN.

Summary updated

Abstract

Accessible mobility is user-conditioned rather than scene-intrinsic, as walkable paths for one embodiment are inaccessible to another. Yet, existing benchmarks evaluate traversability under a fixed and limited embodiment. We present Vision-Language Accessible Navigation (VLAN), containing 1,000 scenes, 20K navigation trajectories, and 160K keyframes across diverse lighting, environmental conditions, scene semantics, and path geometries. For each trajectory, VLAN provides embodiment-specific actions and accessibility features for legged and wheeled agents, including humanoid and quadruped robots and people with visual or mobility impairments, while everyday obstacles and accessibility labels are categorized as Passable, Bypass, HighRisk, and Blocked. Unlike goal-only evaluation, our benchmark measures physical safety and alignment with user-specific constraints via goal-reaching, collision-aware, and accessibility-aligned metrics. We benchmark modern vision-language models and find they often fail to consistently adapt obstacle relevance, path selection, and safety criteria across user profiles, suggesting accessibility-aware navigation requires dedicated datasets, tasks, and metrics beyond conventional embodied navigation evaluation. All data and code open-source at https://github.com/InSAI-Lab/VLAN.
BibTeX
@misc{hu2026vlan,
  author = {Jie Hu and Yu Zheng and Wenzhi Wu and Weixing Feng and Cunlei Cheng and Jiaming Zhang},
  title = {{VLAN: Vision-Language Accessible Navigation}},
  year = {2026},
  eprint = {2611.18346},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18346},
}
Figure