Reaching Is Not Finding: Map-Anchored Visual Prompting for the Last Mile of Aerial Vision-Language Navigation

Ningbo Li, Kebin Liu, Jingcan Chen, Weichen Zhang, Rong Ding, Yunhao Liu, Mo LiarXiv 2026 · arXiv

TL;DR

This paper identifies a critical "reaching-is-not-finding" gap in aerial vision-language navigation, where drones successfully navigate to a landmark but fail to identify the specific target object. The authors introduce MAP-VP, a plug-and-play module that draws map-anchored landmark outlines directly onto the drone's view as visual prompts, roughly doubling the success rates of state-of-the-art navigators by teaching a fine-tuned VLM to use these landmarks as spatial references.

What Problem Does This Paper Address?

Aerial Vision-Language Navigation (VLN) requires a drone to locate a specific object (e.g., "a dark red vehicle") based on its spatial relationship to landmarks (e.g., "parked along Grafton Street across from a dark blue car").

The authors discovered a systemic failure they call the "reaching-is-not-finding" gap. Current navigation models are surprisingly good at reaching the correct neighborhood by following landmark names, but they are terrible at the "last mile." Once the drone arrives, it often stops near the target but fails to pick the correct instance among similar-looking objects. A simple target-swap test proved that most navigators essentially ignore the target description at the end, relying almost entirely on landmark proximity.

Figure 1. Reaching is not finding. A real CityNav test episode. (a) The navigator (HETT) reaches the described car but stops outside the success radius. (b) At the same stop, CityNav’s official goal-selection step, which detects the target phrase with GroundingDINO, picks a look-alike. (c) MAP-VP, applied at the same stop, draws the map landmark onto the view and finds the target. Green: the described target; red: a wrong choice; yellow: the landmark.

Methods

The authors propose MAP-VP (Map-Anchored Visual Prompting), a modular addition that runs after a primary navigator has finished its flight.

  1. Visual Prompting: MAP-VP takes the drone's final top-down view and overlays the outlines and names of landmarks mentioned in the instruction. These polygons are projected from existing map data (like OpenStreetMap).
  2. Fine-tuned VLM: A 4B-parameter Vision-Language Model (Qwen3-VL) is fine-tuned using LoRA to interpret these visual prompts. Crucially, the model learns to treat the yellow outlines as references rather than the answer itself.
  3. Automatic Supervision: Training data is generated automatically from benchmark annotations. The model is trained to output a bounding box for the target or a "null" response if the target is not visible.
  4. Look-Again (LA) Mechanism: If the model returns "null," the drone climbs from 50m to 100m to get a wider field of view and tries again.
  5. Execution: Once a target is identified, the drone performs a final short flight to the center of the predicted box.

Datasets

The study primarily uses CityNav, a large-scale benchmark for aerial VLN featuring real-world city scans of Birmingham and Cambridge. The authors utilized the revised annotations from HETT, which corrected landmark lists and filtered out noisy episodes. Evaluation was conducted on the test-unseen split to ensure the model could generalize to entirely new urban environments.

Key Insights and Ideas

  • Navigators are Landmark-Seekers: A simple "landmark-centroid" rule (flying to the average position of named landmarks) outperformed every trained navigator tested, achieving a 40.2% success rate. This highlights that existing models prioritize landmark proximity over target identification.
  • The Synergy of Anchors and Supervision: Drawing landmarks on the image (anchors) and fine-tuning the model (supervision) are both necessary. Without fine-tuning, zero-shot VLMs tend to mistake the marked landmarks for the target. Without the visual anchors, the model lacks the geometric context to resolve spatial relations like "across from" or "in front of."
  • Spatial Grounding via Attention: Attention analysis showed that when landmarks are drawn, the model's internal attention effectively grounds the landmark names to the specific pixels within the outlines, which then allows it to shift focus to the correct target nearby.

Key Results

MAP-VP provides a massive performance boost across six different navigator architectures without requiring them to be retrained:

  • HETT (State-of-the-art): Success Rate (SR) jumped from 25.9% to 54.6%.
  • FlightGPT (VLM-based): SR increased from 24.2% to 48.9%.
  • Recovery Rate: For episodes where the target was already in the drone's final view but the navigator failed, MAP-VP recovered 50–62% of those failures.
  • Precision: At a stricter 10m success threshold (SR@10), HETT's performance nearly quadrupled, rising from 10.1% to 38.0%.
Figure 2. Performance comparison across navigators. MAP-VP significantly recovers "in-view" failures that other methods miss.

Limitations and Future Work

  • Range Limit: MAP-VP is a "last mile" solution. If the base navigator stops more than 75m away from the target, the target is usually not in view, and MAP-VP cannot help.
  • Counting Weakness: The model still struggles with instructions involving counting (e.g., "the 8th house to the left"). Success rates on counting-based instructions were about 10 points lower than average.
  • Map Dependency: The system assumes access to accurate GPS and landmark polygons. Real-world errors in pose or map data could shift the visual prompts and mislead the model.
  • Future Directions: Integrating this visual prompting logic directly into the navigation phase (rather than just the final step) could potentially help drones that currently fail to even reach the target's vicinity.

Summary updated

Abstract

Aerial vision-language navigation (VLN) asks a drone to find an object described in natural language. The description usually locates the object relative to named landmarks, such as streets and buildings. We observe that current navigators often bring the drone close to the target and then stop at the wrong place: they reach the target without finding it. This gap is shared by navigators of very different designs, from imitation learners to a fine-tuned 7B vision-language model. A simple test reveals the cause. When we replace the target object in the instruction with another one next to the same landmarks, no navigator goes to it, and most stop where they did before. Navigators learn to reach the landmarks, not to identify the described object. We therefore propose MAP-VP, a plug-and-play module for the last step. After any navigator stops, MAP-VP draws the landmarks named in the instruction onto the drone's current view, using the landmark outlines that the benchmark provides on a map. A fine-tuned 4B vision-language model then locates the target relative to these landmarks. The training data are generated automatically from the benchmark annotations. Neither drawing the landmarks nor fine-tuning helps much alone, but together they are highly effective. Without retraining any navigator, MAP-VP significantly improves all six navigators we test on CityNav. It roughly doubles the success rate of the two strongest trained navigators, raising HETT from 25.9% to 54.6% and FlightGPT from 24.2% to 48.9%, whereas CityNav's official detector-based last step adds less than two points to HETT.
BibTeX
@misc{li2026reaching,
  author = {Ningbo Li and Kebin Liu and Jingcan Chen and Weichen Zhang and Rong Ding and Yunhao Liu and Mo Li},
  title = {{Reaching Is Not Finding: Map-Anchored Visual Prompting for the Last Mile of Aerial Vision-Language Navigation}},
  year = {2026},
  eprint = {2611.11510},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11510},
}
Figure