EdgeWorkflowReal: An Edge Computing based Workflow Execution Engine for Smart Systems

Xuejun Li, Ran Ding, Xiao Liu, Jia Xu, Yun Yang, John C. GrundyarXiv 2021 · OpenReview

Abstract

Current cloud-based smart systems suffer from weaknesses such as high response latency, limited network bandwidth and the restricted computing power of smart end devices which seriously affect the system's QoS (Quality of Service). Recently, given its advantages of low latency, high bandwidth and location awareness, edge computing has become a promising solution for smart systems. However, the development of edge computing based smart systems is a very challenging job for software developers who do not have the skills for the creation of edge computing environments. The management of edge computing resources and computing tasks is also very challenging. Workflow technology has been widely used in smart systems to automate task and resource management, but there does not yet exist a real-world deployable edge computing based workflow execution engine. To fill this gap, we present EdgeWorkflowReal, an edge computing based workflow execution engine for smart systems. EdgeWorkflowReal supports: 1) automatic creation of a real edge computing environment according to user settings; 2) visualized modelling of edge workflow applications; and 3) automatic deployment, monitoring and performance evaluation of edge workflow applications in a smart system.

Introduction

While cloud computing has revolutionized data processing, it struggles with "smart" systems—such as autonomous drones or smart factories—that require near-instantaneous response times. Sending data to a distant cloud server introduces high latency and consumes massive bandwidth. Edge computing solves this by processing data closer to the source. However, managing tasks across a heterogeneous mix of devices—ranging from battery-powered UAVs to edge gateways and infinite cloud resources—is incredibly difficult. Existing workflow engines are often built for the cloud or are purely simulators that cannot execute real code. EdgeWorkflowReal bridges this gap by providing a real-world deployable engine designed for edge constraints.

System Architecture

EdgeWorkflowReal utilizes a three-layered architecture to manage the entire lifecycle of an edge application:

  1. User Interface Layer: A web-based portal for visual workflow modeling using Directed Acyclic Graphs (DAGs) and binding them to real computing tasks.
  2. Workflow Management System Layer: The core engine containing a simulator for pre-deployment evaluation and an execution engine for task scheduling, container management, and monitoring.
  3. Database and Resource Layer: A storage layer using MySQL and a Docker-based container pool, ensuring consistent task execution across diverse hardware.
The three-layered architecture of EdgeWorkflowReal, comprising the UI, Management, and Resource layers.

The system supports various scheduling algorithms, including heuristics like First-Come-First-Served (FCFS) and advanced meta-heuristics like Particle Swarm Optimization (PSO) and Genetic Algorithms (GA).

Evaluation and Key Insights

The system was evaluated using the Montage Workflow (a standard scientific benchmark) and a custom task suite (including $\pi$ calculation and string similarity).

A critical feature of EdgeWorkflowReal is its Energy-Aware Scheduling. Unlike cloud-centric engines, it treats energy consumption as a primary objective, which is vital for battery-constrained end devices. It supports tiered offloading strategies: All-in-Cloud, All-in-Edge, and a hybrid Energy-Optimal approach that distributes tasks based on where they are most efficient.

Key Results

Experimental results indicate that the Genetic Algorithm (GA) is the most effective scheduler for reducing execution time, outperforming PSO by approximately 27% for small task sets. While the gap narrows as task volume increases, both meta-heuristics significantly outperform simpler methods like MinMin or MaxMin.

The engine's real-world feasibility was demonstrated through a three-tier UAV delivery system. This scenario proved the engine's ability to manage complex hand-offs between flying drones, local edge stations, and backend cloud servers, effectively balancing the load across the UAV, Edge, and Cloud layers.

A real-world UAV delivery system workflow managed by EdgeWorkflowReal across Cloud, Edge, and UAV layers.

Conclusion and Future Work

EdgeWorkflowReal provides a vital bridge between theoretical edge computing research and practical deployment. Future work will focus on expanding the library of supported custom tasks, testing on a wider variety of ARM-based hardware (e.g., Raspberry Pi, Jetson Nano), and improving robustness against network fluctuations in dynamic mobile environments.

Summary updated

Abstract

Current cloud-based smart systems suffer from weaknesses such as high response latency, limited network bandwidth and the restricted computing power of smart end devices which seriously affect the system's QoS (Quality of Service). Recently, given its advantages of low latency, high bandwidth and location awareness, edge computing has become a promising solution for smart systems. However, the development of edge computing based smart systems is a very challenging job for software developers who do not have the skills for the creation of edge computing environments. The management of edge computing resources and computing tasks is also very challenging. Workflow technology has been widely used in smart systems to automate task and resource management, but there does not yet exist a real-world deployable edge computing based workflow execution engine. To fill this gap, we present EdgeWorkflowReal, an edge computing based workflow execution engine for smart systems. EdgeWorkflowReal supports: 1) automatic creation of a real edge computing environment according to user settings; 2) visualized modelling of edge workflow applications; and 3) automatic deployment, monitoring and performance evaluation of edge workflow applications in a smart system.
BibTeX
@misc{li2021edgeworkflowreal,
  author = {Xuejun Li and Ran Ding and Xiao Liu and Jia Xu and Yun Yang and John C. Grundy},
  title = {{EdgeWorkflowReal: An Edge Computing based Workflow Execution Engine for Smart Systems}},
  year = {2021},
  eprint = {2102.00234},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2102.00234},
}
Figure