Joint Optimization of Service Migration and Resource Allocation in Mobile Edge-Cloud Computing

Zhenli He, Liheng Li, Ziqi Lin, Yunyun Dong, Jianglong Qin, Keqin Li

TL;DR

This paper tackles the challenge of optimizing service migration and resource allocation in mobile edge-cloud computing (MECC) to minimize task response delays and failure rates. They propose a novel deep reinforcement learning (DRL) approach, specifically Advantage Actor-Critic (A2C), that dynamically adapts these strategies, outperforming existing methods in simulations.

What Problem Does This Paper Address?

The rapid growth of the Internet of Things (IoT) and mobile applications has led to the development of Mobile Edge-Cloud Computing (MECC). MECC aims to combine the vast resources of cloud computing with the low latency of edge computing to meet the demands of latency-sensitive applications. However, a significant challenge arises from user mobility and the dynamic nature of computational demands. When users move, their current edge server (ES) might become too distant, necessitating service migration. This migration process, along with resource allocation decisions, needs to be optimized to maintain Quality of Service (QoS).

Existing research often addresses service migration and resource allocation separately or overlooks critical aspects like:

  • Interdependencies: The complex interplay between migration strategies and resource allocation.
  • Migration Delays: The impact of delays introduced by the migration process itself on performance.
  • Incomplete Tasks: Handling tasks that are not fully completed before a migration occurs, which can lead to data loss and compromised QoS.

This paper aims to fill these gaps by developing a unified framework that jointly optimizes service migration and resource allocation, considering these overlooked complexities.

Methods

The core of this paper's contribution lies in its proposed Advantage Actor-Critic (A2C)-based dynamic migration and resource allocation algorithm.

Here's a breakdown of their approach:

  • Problem Formulation: The joint optimization problem is formulated as a Markov Decision Process (MDP). This involves defining the system's state, the actions an agent can take, the transition probabilities between states, and a reward function.
  • State: Represents the real-time status of tasks, including remaining time, size, type, associated user, and current ES location.
  • Action: Encompasses both the decision of where to migrate a service and the amount of computing resources to allocate on the target ES.
  • Reward: Designed to minimize average task response delay. Negative rewards are given for delays and task failures.
  • A2C Algorithm: This is a deep reinforcement learning algorithm that uses two neural networks:
  • Actor Network: Learns the policy (how to choose actions) to maximize expected rewards.
  • Critic Network: Learns the state-value function to estimate the expected future rewards from a given state, aiding the actor's learning process.
  • Handling Incomplete Tasks: The framework explicitly considers scenarios where tasks are not completed before migration, requiring the migration of task data and context.
  • Joint Optimization: Unlike many prior works, this approach simultaneously optimizes migration and resource allocation, recognizing their interdependence.

Datasets

Not discussed in the paper. The authors conducted simulations using a randomly generated MECC system with specified parameters.

Key Insights and Ideas

  • The Power of Joint Optimization: The paper strongly demonstrates that treating service migration and resource allocation as a single, integrated problem yields significantly better results than optimizing them in isolation. This is because these decisions are highly interdependent.
  • DRL for Dynamic Environments: Deep Reinforcement Learning, particularly A2C, proves to be a robust framework for handling the dynamic and complex nature of MECC environments, where states and optimal actions change rapidly due to user mobility and fluctuating demands.
  • Migration Delays Matter: The study highlights that ignoring the delays introduced by the migration process itself can lead to suboptimal decisions and degraded QoS, especially for time-sensitive applications.
  • Handling Incomplete Tasks is Crucial: The ability to manage tasks that are in progress during migration is essential for preventing data loss and maintaining service continuity, a gap addressed by this work.
  • A2C's Efficiency: The A2C algorithm consistently outperforms other benchmark methods in terms of both average response delay and task failure rate, suggesting its effectiveness in balancing these critical performance metrics.

Key Results

The simulation results showcase the superiority of the proposed A2C-based approach:

  • Reduced Average Response Delay: The A2C algorithm consistently achieved the lowest average service response delay across various scenarios, outperforming methods like Follow-Avg, PSO, PPO, and DDPG. For instance, it reduced the average response delay by 0.15s compared to PSO across different numbers of ESs.
  • Lower Task Failure Rate: The A2C approach also demonstrated the lowest task failure rate, maintaining it below 10% even under challenging conditions, whereas other methods experienced rates as high as 20%.
  • Scalability: The algorithm showed good performance even when the network scale expanded to 40 users and 20 ESs, reducing average response delay by 0.1s and average failure rate by 10% compared to other methods.
  • Efficiency: The A2C method also exhibited efficient decision-making, with a significantly shorter decision-making duration per step compared to the PSO method.

Limitations and Future Work

The authors acknowledge several areas for future research:

  • Migration Costs: While the focus has been on response delay, future work should explore strategies to simultaneously reduce migration costs alongside average response times.
  • Task Offloading Strategy: The study primarily examines migration and resource allocation. However, the task offloading strategy also significantly impacts system performance, especially under varying communication conditions. Future research should investigate the complex interplay between offloading, migration, and resource allocation.
  • Data Security and Privacy: The centralized decision-making process raises concerns about user data leakage. Future work needs to explore methods for ensuring data security and privacy without compromising system decision-making efficiency.
  • GPU Acceleration: While the algorithm's overhead is manageable, for larger-scale problems, GPU acceleration could be explored to further reduce training time.

Summary updated

Abstract

In the rapidly evolving domain of mobile edge–cloud computing (MECC), the proliferation of Internet of Things (IoT) devices and mobile applications poses significant challenges, particularly in dynamically managing computational demands and user mobility. Current research has partially addressed aspects of service migration and resource allocation, yet it often falls short in thoroughly examining the nuanced interdependencies between migration strategies and resource allocation, the consequential impacts of migration delays, and the intricacies of handling incomplete tasks during migration. This study advances the discourse by introducing a sophisticated framework optimized through a deep reinforcement learning (DRL) strategy, underpinned by a Markov decision process (MDP) that dynamically adapts service migration and resource allocation strategies. This refined approach facilitates continuous system monitoring, adept decision making, and iterative policy refinement, significantly enhancing operational efficiency and reducing response times in MECC environments. By meticulously addressing these previously overlooked complexities, our research not only fills critical gaps in the literature but also enhances the practical deployment of edge computing technologies, contributing profoundly to both theoretical insights and practical implementations in contemporary digital ecosystems.
BibTeX
@article{he2024joint,
  author = {Zhenli He and Liheng Li and Ziqi Lin and Yunyun Dong and Jianglong Qin and Keqin Li},
  title = {{Joint Optimization of Service Migration and Resource Allocation in Mobile Edge-Cloud Computing}},
  journal = {Algorithms},
  year = {2024},
  doi = {10.3390/a17080370},
}
Figure