Fuzzy-Assisted Mobile Edge Orchestrator and SARSA Learning for Flexible Offloading in Heterogeneous IoT Environment

Tran Trong Khanh, Tran Hoang Hai, Md. Delowar Hossain, Eui-Nam HuhSensors 2022 · DOI

TL;DR

This paper introduces Fu-SARSA, a novel two-stage algorithm that combines fuzzy logic and SARSA reinforcement learning to optimize task offloading in heterogeneous IoT environments. By intelligently deciding where to offload tasks (local edge, cloud, or neighboring edge servers), Fu-SARSA significantly reduces task failure rates and service times, especially under network congestion.

What Problem Does This Paper Address?

The proliferation of Internet of Things (IoT) devices in 5G networks has led to an explosion of data and demanding applications. These applications require low latency and substantial computational resources, which resource-constrained IoT devices often lack. While mobile cloud computing (MCC) and multi-access edge computing (MEC) offer solutions by offloading tasks to more powerful infrastructure, existing approaches face challenges. These include unbalanced loads across edge servers, high task failure rates due to network unpredictability and limited edge server capacity, and the difficulty of making optimal offloading decisions in dynamic environments. Specifically, previous work often overlooked the potential of utilizing neighboring edge servers when local ones are overloaded, leading to suboptimal resource utilization and performance.

Methods

The proposed solution, Fu-SARSA, is a two-stage algorithm:

  1. Fuzzy Logic Phase (Stage 1): This stage acts as an initial filter, using fuzzy logic to make a preliminary decision on whether a task should be offloaded to a local edge server, a cloud server, or a neighboring edge server. It considers input parameters like WLAN/MAN delay, local/remote edge VM utilization. If the decision is to offload to a local edge server, the process ends. Otherwise, it proceeds to the second stage.
  2. SARSA Reinforcement Learning Phase (Stage 2): If the task is slated for offloading to a neighboring edge server, this stage employs the SARSA (State-Action-Reward-State-Action) reinforcement learning algorithm. SARSA is used to intelligently select the best neighboring edge server among available options, considering factors like task size, MAN delay, average VM utilization, and delay sensitivity. SARSA learns an optimal policy by interacting with the MEC network environment, aiming to maximize cumulative rewards (which are designed to penalize task failures and high latency).

The Mobile Edge Orchestrator (MEO) acts as the central decision-maker, managing the network topology and orchestrating the offloading decisions based on the Fu-SARSA algorithm.

Datasets

The paper does not explicitly mention training datasets. Instead, it uses a simulation environment called EdgeCloudSim to model and evaluate the proposed algorithm. The simulation parameters are based on real-world properties for WLAN and WAN, and a single-server queue model for MAN delay.

Key Insights and Ideas

  • Leveraging Neighboring Edge Servers: A crucial insight is the underutilization of neighboring edge servers. Fu-SARSA explicitly addresses this by using SARSA to find the best available neighboring server when local resources are exhausted, leading to better load balancing.
  • Two-Stage Decision Making: The combination of fuzzy logic for initial broad decisions and SARSA for fine-grained selection of the optimal neighboring server provides a robust and adaptive approach to complex offloading scenarios.
  • Adaptability to Dynamic Environments: Fuzzy logic's ability to handle imprecise and uncertain information, coupled with SARSA's reinforcement learning capabilities, makes the system highly adaptable to the constantly changing nature of MEC networks.
  • Application-Specific Offloading: The study considers four distinct application types (healthcare, AR, infotainment, compute-intensive) with varying characteristics (delay sensitivity, resource needs), highlighting that a one-size-fits-all offloading strategy is insufficient.

Key Results

The Fu-SARSA algorithm demonstrated superior performance compared to other approaches (utilization, OWB, fuzzy-based competitor, hybrid) across several metrics:

  • Reduced Task Failure Rate: Fu-SARSA significantly lowered the average task failure rate, especially under high network load (e.g., 2000-2500 IoT devices). It was particularly effective for healthcare and AR applications.
  • Improved Service Time: The algorithm achieved the best results in terms of average service time, outperforming others by effectively considering network conditions and task properties.
  • Efficient VM Utilization: While Fu-SARSA utilized more CPU resources due to successful offloading of heavy tasks, this contributed to better overall system performance and reduced failures.
  • Robustness under Overload: The proposed method showed remarkable resilience and efficiency when the network became congested, a scenario where other methods struggled.

Limitations and Future Work

  • Limitations: The paper acknowledges that the performance of Fu-SARSA is dependent on the chosen parameters for fuzzy logic and SARSA (e.g., learning rate, epsilon). While they investigated some values, further optimization might be possible. The current model focuses on specific types of applications and network configurations.
  • Future Work: The authors suggest applying deep learning techniques for even more efficient offloading to further enhance system performance. They also plan to explore more complex network scenarios and a wider range of application types.

Summary updated

BibTeX
@article{khanh2022fuzzy,
  author = {Tran Trong Khanh and Tran Hoang Hai and Md. Delowar Hossain and Eui-Nam Huh},
  title = {{Fuzzy-Assisted Mobile Edge Orchestrator and SARSA Learning for Flexible Offloading in Heterogeneous IoT Environment}},
  journal = {Sensors},
  year = {2022},
  doi = {10.3390/s22134727},
}
Figure