Offloading Revenue Maximization in Multi-UAV-Assisted Mobile Edge Computing for Video Stream

Bin Li, Huimin Shan

Abstract

Real-time high-definition video streaming presents significant computational challenges for mobile devices, often leading to high latency and rapid battery depletion. While Mobile Edge Computing (MEC) offers a solution by offloading tasks to edge servers, traditional stationary base stations lack the flexibility required for dynamic environments. This paper proposes a collaborative offloading framework that utilizes both Unmanned Aerial Vehicles (UAVs) and idle User Devices (UDs) as mobile edge nodes. To address the incentive problem—encouraging nodes to share resources—the authors formulate a revenue maximization problem. Using a Twin Delayed Deep Deterministic policy gradient (TD3) algorithm, the framework jointly optimizes UAV trajectories, resource pricing, and dynamic video transcoding. Simulation results demonstrate that the TD3-based approach significantly outperforms baseline reinforcement learning models in terms of system-wide revenue and stability.

Introduction

The proliferation of high-definition video applications has placed an unprecedented strain on the computational capabilities of mobile devices. Processing these streams in real-time often leads to excessive latency and rapid battery depletion. Mobile Edge Computing (MEC) has emerged as a promising paradigm to alleviate these issues by offloading intensive tasks to nearby servers. However, deploying fixed MEC infrastructure is costly and often impractical in temporary or high-mobility scenarios.

Unmanned Aerial Vehicles (UAVs) equipped with MEC servers provide a flexible, on-demand alternative. Despite their mobility, UAVs are constrained by finite energy reserves and computing power. Furthermore, most existing research overlooks the economic motivation: why would a UAV operator or a bystander with an idle smartphone provide computing services? This paper addresses this gap by introducing an incentive-driven model for revenue maximization in a multi-tier edge environment.

System Architecture and Incentive Mechanism

The proposed framework considers a three-tier offloading architecture:

  1. Local Processing: The "busy" user device (UD) processes the task locally.
  2. D2D Offloading: Busy UDs offload tasks to nearby "idle" UDs via Device-to-Device (D2D) communication.
  3. UAV Offloading: Tasks are offloaded to aerial MEC servers mounted on UAVs.

The core of the model is the Offloading Revenue Maximization problem. The authors define utility functions for all participants. Revenue is generated through the sale of computing cycles, while costs are derived from energy consumption (both for computation and UAV propulsion) and "inconvenience factors" for idle users. By balancing these factors, the system ensures that service providers are compensated for their resource expenditure.

Methodology: The TD3 Approach

The dynamic nature of the environment—characterized by moving UAVs and varying task loads—is modeled as a Markov Decision Process (MDP). To solve this, the researchers employ the Twin Delayed Deep Deterministic policy gradient (TD3) algorithm.

TD3 improves upon standard Deep Reinforcement Learning (DRL) methods like DDPG by addressing the issue of Q-value overestimation. It utilizes "twin" critic networks and delayed policy updates to ensure more stable convergence. The algorithm manages a high-dimensional action space, including:

  • Offloading Ratios: Determining the fraction of a task to be processed at each tier.
  • Resource Allocation: Managing CPU cycles and transmission power.
  • UAV Trajectories: Dynamically moving UAVs toward user clusters to minimize communication distance and power loss.
  • Dynamic Transcoding: Adjusting video bitrates to optimize processing overhead relative to channel conditions.

Performance Evaluation

The framework was evaluated through numerical simulations in a $200 \times 200$ meter area with 20 UDs and 5 UAVs. Tasks were modeled as video streams ranging from 1.5 to 3.5 Mb.

Comparative Analysis

The TD3 algorithm was compared against Proximal Policy Optimization (PPO) and Deep Deterministic Policy Gradient (DDPG). The results indicated that TD3 achieved a higher average episode reward and more consistent revenue growth. While PPO and DDPG were able to learn the environment, they frequently fell into sub-optimal policies, whereas TD3’s double Q-network architecture allowed it to find more efficient resource configurations.

Scalability and Resource Impact

The study found that system revenue scales positively with the density of service providers. Adding more UAVs or idle UDs consistently increased total revenue, validating the incentive model's effectiveness in a crowded network. Furthermore, increasing the UAVs' computing capacity (from 10 GHz to 30 GHz) led to a direct increase in revenue, as more tasks could be offloaded from local devices, reducing the high "inconvenience costs" associated with local processing.

Conclusion

This research demonstrates that integrating UAVs and D2D communication into a unified MEC framework can significantly enhance the efficiency of real-time video processing. By framing the problem as one of revenue maximization and employing the robust TD3 algorithm, the authors provide a viable path for incentivizing resource sharing in mobile networks. Future work will focus on transitioning from a centralized controller to a multi-agent distributed system and incorporating more complex urban obstacles into the simulation environment.

Summary updated

Abstract

Traditional video transmission systems assisted by multiple uncrewed aerial vehicles (UAVs) are often limited by computing resources, making it challenging to meet the demands for efficient video processing. To solve this challenge, this article presents a multi-UAV-assisted device-to-device (D2D) mobile edge computing system for the maximization of task offloading profits in video stream transmission. In particular, the system enables UAVs to collaborate with idle user devices to process video computing tasks by introducing D2D communications. To maximize the system efficiency, the article jointly optimizes power allocation, video transcoding strategies, computing resource allocation, and UAV trajectory. The resulting nonconvex optimization problem is formulated as a Markov decision process (MDP) and solved relying on the twin delayed deep deterministic policy gradient (TD3) algorithm. Numerical results indicate that the proposed TD3 algorithm performs a significant advantage over other traditional algorithms in enhancing the overall system efficiency.
BibTeX
@article{li2024offloading,
  author = {Bin Li and Huimin Shan},
  title = {{Offloading Revenue Maximization in Multi-UAV-Assisted Mobile Edge Computing for Video Stream}},
  journal = {IEEE Internet of Things Journal},
  year = {2024},
  doi = {10.1109/jiot.2024.3513409},
}
Figure