TL;DR
This paper introduces Astertower, a lightweight microservice composition platform designed for cloud-edge integration in industrial IoT scenarios. It addresses the resource limitations of edge devices and the complexity of traditional cloud-native workflows by proposing a novel architecture that leverages Kubernetes Operators and a custom lightweight workflow engine (Astermule) to significantly reduce execution time and unify data flow.
What Problem Does This Paper Address?
The rapid adoption of cloud-native technologies and microservices has brought benefits, but their application in industrial Internet of Things (IIoT) scenarios, especially for small and medium-sized enterprises (SMEs), faces significant hurdles. These include:
- Resource Constraints at the Edge: Traditional cloud-native workflows are resource-intensive and unsuitable for edge devices with limited computational power and memory.
- Complexity in Digital Transformation: SMEs struggle with integrating diverse industrial equipment, managing complex workflows, and the slow pace of development and deployment.
- Deployment and Maintenance Challenges: Microservice applications are often difficult to deploy and maintain, requiring specialized expertise and significant personnel, which can be unaffordable for SMEs.
- Cloud-Edge Integration Gaps: Bridging the communication and data flow between cloud microservices and edge devices, which often use different protocols and have varying capabilities, remains a challenge.
This paper aims to solve the core problem of microservice composition in cloud-edge scenarios, particularly for industrial applications, by proposing a lightweight and efficient solution.
Methods
The proposed solution, named Astertower, is a cloud-edge combined lightweight microservice composition platform. Its core components and novel approaches include:
- Lightweight Workflow Engine (Astermule): This engine is designed to minimize overhead. Instead of starting a separate container for each workflow node (as in traditional CICD workflows), Astermule uses a single-process model with Goroutines and Go channels. This allows for concurrent execution of nodes within a single Pod, significantly reducing resource consumption and startup/destruction times.
- Kubernetes Operator (Astertower Controller): This component leverages Kubernetes' Custom Resource Definition (CRD) to manage the lifecycle of workflows. It orchestrates the deployment of workflow nodes (as Deployments and Services) and the Astermule engine itself. The controller monitors the state of these resources and manages their transitions through defined phases (Initialized, Waited, Ready, Success, etc.).
- Cloud-Edge Integration: The architecture facilitates seamless integration by synchronizing edge device information into Kubernetes custom resources. The Astermule workflow engine then unifies data flow between cloud services and edge devices, addressing the data gap.
- Business Process Model and Notation (BPMN) 2.0: The workflow design framework is based on BPMN, providing a standardized and intuitive way for users to model business processes.
- Separation of Concerns: The platform separates the control plane (Astertower) from the data plane (Astermule workflow engines), allowing for independent scaling and management.
Datasets
Not discussed in the paper. The experiments were conducted using a simulated Kubernetes cluster on an Alibaba Cloud server.
Key Insights and Ideas
- Goroutines and Channels for Lightweight Execution: The most significant insight is the use of Goroutines and Go channels within a single process for workflow execution. This drastically reduces the overhead associated with container orchestration for each node, making it ideal for resource-constrained edge environments.
- Kubernetes CRDs for Workflow Orchestration: Leveraging Kubernetes CRDs as the orchestration unit for workflows simplifies management, enhances usability, and improves reliability by integrating with the mature Kubernetes ecosystem.
- Unified Data Flow for Cloud-Edge: The Astermule engine's ability to unify data flow between cloud and edge devices is crucial for bridging the gap and enabling consistent data processing across distributed environments.
- Significant Performance Gains: The approach demonstrably outperforms traditional container-native workflow engines like Argo-Workflow, especially as the number of workflow nodes increases, highlighting the effectiveness of the lightweight design.
Key Results
The paper presents experimental results comparing Astertower and Astermule against Argo-Workflow. The key findings are:
- Reduced Execution Time: Astertower and Astermule consistently achieved significantly shorter workflow execution times compared to Argo-Workflow across various scenarios (4, 6, and 10 nodes, with 1, 3, and 5 concurrent workflows).
- Scalability Advantage: The performance advantage of Astertower and Astermule becomes more pronounced as the number of nodes in the workflow increases. For instance, with 10 nodes and 5 concurrent workflows, Astertower and Astermule took 366.149s, while Argo-Workflow took 1037.808s.
- Efficiency in Resource Utilization: While not explicitly quantified with metrics like CPU/memory usage, the design philosophy and comparative results strongly suggest a much more efficient use of resources, particularly at the edge.
Limitations and Future Work
- Simulated Environment: The experiments were conducted on a single-node simulated Kubernetes cluster. Real-world deployment on diverse edge hardware and network conditions might reveal different performance characteristics.
- Limited Scope of Testing: While three workflow sizes and three concurrency levels were tested, a broader range of complex industrial scenarios and edge device types could provide more comprehensive validation.
- Future Work: The authors suggest further optimization and updates to conventional approaches, including power allocation, computation offloading/allocation, handover, and subcarrier allocation, to support future environment requirements in mobile-edge computing networks. They also highlight the need for more robust error handling and fault tolerance mechanisms.
Summary updated