LogisticsVLN: Vision-Language Navigation for Low-Altitude Terminal Delivery Based on Agentic UAVs

Xinyuan Zhang, Yonglin Tian, Fei Lin, Yue Liu, Jing Ma, Xuan Wang, Kornélia Sára Szatmáry, Fei-Yue Wang

TL;DR

This paper introduces LogisticsVLN, a novel system for autonomous UAV-based terminal delivery to specific building windows, leveraging lightweight multimodal large language models (MLLMs). To enable this, they developed the VLD dataset within the CARLA simulator, addressing the gap in current UAV navigation research which focuses on coarse-grained, long-range goals.

What Problem Does This Paper Address?

The explosion of e-commerce has created a demand for efficient and precise last-mile delivery. While autonomous delivery via Unmanned Aerial Vehicles (UAVs) holds great promise, current Vision-Language Navigation (VLN) research for UAVs primarily focuses on long-distance, broad navigation tasks. These approaches are ill-suited for the nuanced requirements of terminal delivery, which often involves precise localization to specific points like a window. Existing VLN systems also often rely on extensive training data or pre-built maps, making them less adaptable to new environments. This paper aims to bridge this gap by developing a system capable of fine-grained, window-level aerial delivery using foundation models without prior environmental knowledge.

Methods

LogisticsVLN employs a modular pipeline built around lightweight multimodal large language models (MLLMs). The system breaks down the delivery process into several key stages:

  • Request Understanding: A Large Language Model (LLM) interprets natural language delivery requests, extracting crucial information like the target floor and a distinctive object for visual identification.
  • Floor Localization: A specialized Floor Count Visual-Language Model (VLM) estimates the drone's current floor level and guides it to ascend to the correct floor. This module is designed to be robust even when a single drone view might span multiple floors.
  • Object Exploration: Once at the target floor, the drone explores the building's exterior. This involves:
  • Object Recognition VLM: Identifies the target window based on the request's description.
  • Viewpoint Selection: A depth-based algorithm intelligently chooses the next best camera view to maximize the chances of spotting the target window, moving beyond simple random or default strategies.
  • Choice VLM & Depth Assistant: These components work together to determine the drone's next action (direction and distance), ensuring safe navigation by considering depth information and avoiding collisions.
  • Safety: A depth assistant module is integrated throughout to ensure operational safety.

Datasets

The authors introduce the Vision-Language Delivery (VLD) dataset, specifically designed for this new aerial terminal delivery task. This dataset was constructed within the CARLA simulator, a realistic 3D environment. The VLD dataset features:

  • 300 tasks across 22 different buildings.
  • A diverse range of target object categories (tools, containers, furniture, etc.).
  • Various building types (low-rise residential, high-rise, villas).
  • A distribution of task difficulties (Easy, Moderate, Hard) defined by the number of turns required to find the target.
  • Linguistically diverse user requests generated using GPT-4o and refined by human experts.

This dataset aims to fill a gap in existing VLN benchmarks, which often lack the focus on precise, continuous aerial navigation required for terminal delivery.

Key Insights and Ideas

  • Foundation Models for Precision Delivery: The paper demonstrates that lightweight MLLMs can be effectively leveraged for complex, fine-grained tasks like window-level delivery, moving beyond their typical use in more general navigation.
  • Modular Design for Adaptability: The modular pipeline allows for subtask-level analysis and potential improvements to individual components, offering valuable insights for deploying foundation models in real-world scenarios.
  • Importance of Viewpoint Selection: A sophisticated, depth-based viewpoint selection strategy significantly outperforms random or default approaches, highlighting the need for intelligent exploration in complex environments.
  • Addressing VLM Limitations: The work directly tackles known VLM challenges like floor counting reliability and object recognition accuracy, proposing novel methods to mitigate these issues.

Key Results

The LogisticsVLN system, when evaluated on the VLD dataset using the Qwen2-VL-7B-Instruct VLM, achieved a Success Rate (SR) of 54.7% and a Success Weighted by Path Length (SPL) of 52.4%.

  • VLM Performance: Qwen2-VL-7B-Instruct performed best among the tested lightweight VLMs, outperforming LLaMA-3.1-11B-Vision-Instruct and Yi-VL-6B. The authors attribute performance differences to object recognition accuracy and floor localization reliability.
  • Floor Localization: The proposed Floor Localization method significantly reduced floor localization failure rates from 61.6% (direct count) to 27.9%.
  • Viewpoint Selection: The proposed Viewpoint Selection algorithm led to noticeable improvements in SR and SPL compared to random or default strategies.
  • Choice VLM Impact: While removing the Choice VLM showed only minor performance drops in some scenarios, ablation studies suggest its critical role in enabling safe exploration in complex architectural designs where depth information alone might be insufficient.

Limitations and Future Work

Limitations:

  • The performance of the system is still limited, with success rates below 60% for the best-performing VLM.
  • The paper acknowledges that Yi-VL frequently refused to provide precise answers in the Floor Count module, indicating ongoing challenges with VLM instruction following.
  • LLaMA-3.1 and Yi-VL showed higher rates of misidentification in object recognition, often due to over-reliance on specific attributes like color.

Future Work:

  • Optimizing the system architecture to better leverage MLLM capabilities.
  • Extending LogisticsVLN to real-world aerial delivery applications.
  • Further research into improving VLM reliability for object recognition and floor localization.

Summary updated

Abstract

The growing demand for intelligent logistics, particularly fine-grained terminal delivery, underscores the need for autonomous UAV (Unmanned Aerial Vehicle)-based delivery systems. However, most existing last-mile delivery studies rely on ground robots, while current UAV-based Vision-Language Navigation (VLN) tasks primarily focus on coarse-grained, long-range goals, making them unsuitable for precise terminal delivery. To bridge this gap, we propose LogisticsVLN, a scalable aerial delivery system built on multimodal large language models (MLLMs) for autonomous terminal delivery. LogisticsVLN integrates lightweight Large Language Models (LLMs) and Visual-Language Models (VLMs) in a modular pipeline for request understanding, floor localization, object detection, and decision-making. To support research and evaluation in this new setting, we construct the Vision-Language Delivery (VLD) dataset within the CARLA simulator. Experimental results on the VLD dataset showcase the feasibility of the LogisticsVLN system. In addition, we conduct subtask-level evaluations of each module of our system, offering valuable insights for improving the robustness and real-world deployment of foundation model-based vision-language delivery systems.
BibTeX
@misc{zhang2025logisticsvln,
  author = {Xinyuan Zhang and Yonglin Tian and Fei Lin and Yue Liu and Jing Ma and Xuan Wang and Kornélia Sára Szatmáry and Fei-Yue Wang},
  title = {{LogisticsVLN: Vision-Language Navigation for Low-Altitude Terminal Delivery Based on Agentic UAVs}},
  year = {2025},
  eprint = {2505.03460},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.03460},
}
Figure