Abstract
The rapid advancement of Low-Altitude Economy Networks (LAENets) has enabled a variety of applications, including aerial surveillance, environmental sensing, and semantic data collection. To support these scenarios, unmanned aerial vehicles (UAVs) equipped with onboard vision-language models (VLMs) offer a promising solution for real-time multimodal inference. However, ensuring both inference accuracy and communication efficiency remains a significant challenge due to limited onboard resources and dynamic network conditions. In this paper, we first propose a UAV-enabled LAENet system model that jointly captures UAV mobility, user-UAV communication, and the onboard visual question answering (VQA) pipeline. Based on this model, we formulate a mixed-integer non-convex optimization problem to minimize task latency and power consumption under user-specific accuracy constraints. To solve the problem, we design a hierarchical optimization framework composed of two parts: (i) an Alternating Resolution and Power Optimization (ARPO) algorithm for resource allocation under accuracy constraints, and (ii) a Large Language Model-augmented Reinforcement Learning Approach (LLaRA) for adaptive UAV trajectory optimization. The large language model (LLM) serves as an expert in refining reward design of reinforcement learning in an offline fashion, introducing no additional latency in real-time decision-making. Numerical results demonstrate the efficacy of our proposed framework in improving inference performance and communication efficiency under dynamic LAENet conditions.
Introduction
As we transition toward a robust "Low-Altitude Economy," Unmanned Aerial Vehicles (UAVs) are evolving from simple remote-controlled aircraft into intelligent agents capable of complex perception and reasoning. Integrating Vision-Language Models (VLMs) directly onto UAVs enables sophisticated tasks such as Visual Question Answering (VQA). For instance, a drone could autonomously identify specific objects or anomalies in a crowded urban environment based on natural language queries from ground users.
However, deploying VLMs on resource-constrained UAVs presents a severe "accuracy-efficiency" trade-off. High-resolution imagery is essential for accurate inference but imposes significant transmission delays and computational burdens. Furthermore, the UAV's physical trajectory dynamically alters signal strength and energy consumption. Addressing these challenges requires a holistic optimization of communication, computation, and flight paths—a problem that is mathematically NP-hard and highly non-convex.
Methodology
To tackle the joint optimization of resource allocation and trajectory control, we propose a hierarchical framework that decomposes the original complex problem into manageable sub-problems.
Hierarchical Optimization Framework
The core of our approach lies in separating "fast" variables (transmission power and image resolution) from "slow" variables (UAV trajectory). As illustrated in the framework overview, the original optimization problem $\mathbb{P}_0$ is decomposed into resource allocation ($\mathbb{P}_1$) and trajectory control ($\mathbb{P}_3$).
ARPO: Alternating Resolution and Power Optimization
The ARPO algorithm handles the immediate resource allocation for each user. It utilizes a Branch and Bound (B&B) algorithm to select optimal discrete image resolutions based on empirical lookup tables that map resolution to expected accuracy and inference latency. Once resolutions are fixed, the algorithm applies Karush-Kuhn-Tucker (KKT) conditions to optimize user transmit power, ensuring efficient uplink communication.
LLaRA: LLM-augmented Reinforcement Learning Approach
Trajectory planning is a long-horizon problem addressed through Deep Reinforcement Learning (DRL). A novel contribution of this work is the use of a Large Language Model (LLM), specifically GPT-4o, as an offline reward designer.
Instead of manual reward engineering, the LLM acts as an expert that receives the system model and iteratively refines Python code for the DRL reward function based on training performance. This "LLM-in-the-loop" approach allows for sophisticated reward structures, such as "tail-aware" rewards that prioritize users with the longest wait times, without adding any computational latency during real-time flight operations.
Key Insights and Results
Our evaluation using benchmarks like TextVQA and empirical profiling of models such as LLaVA-1.5-13B and LLaVA-HR-7B yielded several critical insights:
- Resolution vs. Model Size: We found that optimizing image resolution is often more impactful than increasing model parameters. A smaller, high-resolution model (e.g., LLaVA-HR-7B) frequently outperforms much larger models (e.g., Qwen-225B) when the latter is restricted to low-resolution inputs due to bandwidth constraints.
- Latency Reduction: The ARPO-LLaRA framework achieved a 45% reduction in task latency compared to random policies and a 25% improvement over standard geometric heuristics.
- LLM-Enhanced RL Performance: The LLM-designed reward function provided a 13.7% improvement in latency reduction over traditional DRL baselines with human-designed rewards, demonstrating the LLM's capability as a meta-optimizer.
- Accuracy Maintenance: By intelligently balancing resolution and power, the system maintained inference accuracy above 67% while significantly optimizing the communication uplink.
Conclusion
This work demonstrates that hierarchical optimization, combined with the reasoning capabilities of LLMs for offline reward design, can effectively bridge the gap between complex VLM requirements and the limited resources of UAV-enabled networks. Future research will explore multi-UAV coordination and the impact of highly mobile ground users on system performance.
Summary updated