AIR-VLA: Vision-Language-Action Systems for Aerial Manipulation

Jianli Sun, Bin Tian, Qiyao Zhang, Chengxiang Li, Zihan Song, Zhiyong Cui, Yisheng Lv, Yonglin Tian

Abstract: While Vision-Language-Action (VLA) models have achieved remarkable success in ground-based embodied intelligence, their application to Aerial Manipulation Systems (AMS) remains a largely unexplored frontier. The inherent characteristics of AMS, including floating-base dynamics, strong coupling between the UAV and the manipulator, and the multi-step, long-horizon nature of operational tasks, pose severe challenges to existing VLA paradigms designed for static or 2D mobile bases. To bridge this gap, we propose AIR-VLA, the first VLA benchmark specifically tailored for aerial manipulation. We construct a physics-based simulation environment and release a high-quality multimodal dataset comprising 3000 manually teleoperated demonstrations, covering base manipulation, object & spatial understanding, semantic reasoning, and long-horizon planning. Leveraging this platform, we systematically evaluate mainstream VLA models and state-of-the-art VLM models. Our experiments not only validate the feasibility of transferring VLA paradigms to aerial systems but also, through multi-dimensional metrics tailored to aerial tasks, reveal the capabilities and boundaries of current models regarding UAV mobility, manipulator control, and high-level planning. AIR-VLA establishes a standardized testbed and data foundation for future research in general-purpose aerial robotics.

1. Introduction

Most current research in "embodied AI" is confined to ground-based platforms. Models like RT-1 or OpenVLA have demonstrated proficiency in tabletop manipulation and 2D navigation, yet they encounter significant hurdles when transitioned to Aerial Manipulation Systems (AMS)—unmanned aerial vehicles (UAVs) equipped with robotic manipulators.

Aerial manipulation represents a "boss level" challenge for robotics due to three primary factors:

  1. Floating-Base Dynamics: Unlike stationary or heavy ground-based robots, a drone's base reacts to the movement of its arm. This kinematic coupling makes high-precision tasks exceptionally difficult.
  2. 3D Complexity: The operational environment is a vast, three-dimensional volume, necessitating advanced spatial reasoning far beyond what is required for flat-surface tasks.
  3. Data Scarcity: High-quality datasets for complex aerial manipulation are virtually non-existent compared to the abundance of data available for ground-based systems.

2. The AIR-VLA Framework and Dataset

To address these challenges, the authors introduce the AIR-VLA framework, a comprehensive platform for training and evaluating aerial VLA models. The framework integrates a high-fidelity simulation environment with a robust data pipeline.

Overview of the AIR-VLA framework, showing the transition from 3000 manually collected data records to the VLA Model architecture.

Simulation and Hardware

The environment is built on NVIDIA Isaac Sim using the PhysX 5 engine, simulating realistic fluid dynamics (such as wind gusts) and lighting to minimize the "Sim-to-Real" gap. The hardware stack models a quadrotor UAV paired with a 7-Degree-of-Freedom (DoF) Franka Panda manipulator, creating a complex 12-DoF system.

The AIR-VLA Dataset

The core contribution is the AIR-VLA Dataset, which includes 3000 expert demonstrations manually teleoperated across 60 distinct tasks. The dataset is organized into four task suites:

  • Base Manipulation: Fundamental actions like picking, placing, and twisting.
  • Object & Spatial Understanding: Recognizing colors and shapes within a 3D volume.
  • Semantic Reasoning: Following complex, implicit natural language instructions.
  • Long-Horizon Planning: Multi-step missions, such as opening a drawer to retrieve and place an object.

Each record in the dataset provides multimodal inputs, including 4-view RGB-D video, proprioceptive data (joint angles, UAV velocity), and language instructions augmented by Large Language Models (LLMs) for linguistic diversity.

3. Experimental Evaluation and Insights

The researchers systematically evaluated mainstream VLA and VLM models using a weighted scoring system ($S_{total}$) that accounts for base positioning ($S_{pos}$), manipulator efficacy ($S_{arm}$), environmental safety ($S_{safe}$), and task progression ($S_{task}$).

A sequence of frames illustrating a multi-step aerial manipulation task, highlighting the complexity of 3D spatial reasoning and long-horizon planning.

Key Findings

  • The Precision Bottleneck: While VLA models excel at coarse navigation to a target area, they struggle with the "last centimeter" of manipulation. The floating base effectively makes the target a moving goalpost for the arm.
  • Foundation Model Adaptability: Large-scale pre-trained models (such as the $\pi$ series) adapt rapidly to aerial tasks. With as few as 30–50 fine-tuning demonstrations, these models outperform traditional imitation learning methods like ACT or Diffusion Policy. The $\pi_{0.5}$ model emerged as the top performer with an overall score of 42.0.
  • VLMs as High-Level Planners: Vision-Language Models (VLMs) like Qwen3-VL-8B-Instruct act as effective high-level "brains," breaking down complex missions into logical sub-tasks with a success rate of 23.3% on difficult tasks.
  • Perspective and Spatial Gaps: Models show a heavy dependency on fixed third-person views; performance drops significantly when relying solely on onboard cameras. Furthermore, even advanced VLMs struggle with precise 3D spatial navigation (e.g., identifying "front-left" approach vectors).

4. Conclusion and Future Directions

AIR-VLA establishes a critical foundation for general-purpose aerial robotics. However, several frontiers remain. Future work must address the Sim-to-Real transition, where real-world aerodynamics and sensor noise are more unpredictable. Additionally, moving from passive perception to active vision—where the drone autonomously adjusts its viewpoint for better task execution—is essential. Finally, integrating explicit multi-body physics priors into end-to-end learning models could better manage the complex coupling between the UAV base and its manipulator.

Summary updated

BibTeX
@inproceedings{sun2026air,
  author = {Jianli Sun and Bin Tian and Qiyao Zhang and Chengxiang Li and Zihan Song and Zhiyong Cui and Yisheng Lv and Yonglin Tian},
  title = {{AIR-VLA: Vision-Language-Action Systems for Aerial Manipulation}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=NuR4lG4gKB},
}
Figure