TEMPOS: QoS Management Middleware for Edge Cloud Computing FaaS in the Internet of Things

Andrea Garbugli, Andrea Sabbioni, Antonio Corradi, Paolo BellavistaIEEE Access 2022 · OpenReview

Abstract

Several classes of advanced Internet of Things (IoT) applications, such as those in industrial manufacturing, require strict Quality of Service (QoS) management to guarantee performance indicators. These applications often face "stochastic noise" in real-world deployments, ranging from limited bandwidth to resource contention in virtualized environments. This paper introduces TEMPOS, a novel IoT-oriented middleware that coordinates QoS monitoring, control, and management across networking and processing resources. By targeting edge cloud environments and the Serverless (Function-as-a-Service) paradigm, TEMPOS demonstrates that effective QoS differentiation is achievable through the holistic coordination of existing technologies.

Introduction: The Challenge of QoS in Edge FaaS

As the Internet of Things (IoT) moves into industrial manufacturing and smart infrastructure, applications demand more than just connectivity—they require performance guarantees. While the Serverless paradigm, specifically Function-as-a-Service (FaaS), is popular for its scalability and ease of use, it is traditionally a "best-effort" model.

In a typical FaaS setup, developers have little control over when a function actually executes or how long a message sits in a network queue. At the "Edge"—where computing resources are closer to sensors but more limited than in massive data centers—this lack of control leads to "stochastic noise." Network congestion and CPU contention make it nearly impossible to guarantee the low latency and minimal jitter required by industrial robots or augmented reality systems. TEMPOS aims to bridge this gap by managing the entire "invocation stack" from the moment a sensor triggers an event to the completion of function processing.

The TEMPOS Architecture

TEMPOS (Time-Effective Middleware for Priority Oriented Serverless) is designed around three functional "slices" and three conceptual layers (Logical, QoS, and System). This structure allows for the holistic coordination of system-level technologies to ensure end-to-end performance.

  1. Bridging Slice: Manages "Triggers" that convert external IoT events into internal system events.
  2. Delivery Slice: Utilizes a custom Message-Oriented Middleware (MOM) with priority queues to ensure high-priority messages are not delayed by best-effort traffic.
  3. Processing Slice: Features an "Invoker" that leverages Linux real-time scheduling to allow high-priority functions to preempt lower-priority tasks.
TEMPOS system architecture showing the invocation flow: (a) direct invocation and (b) invocation via Message-Oriented Middleware (MOM).

The system integrates several key technologies to achieve its goals:

  • Time-Sensitive Networking (TSN): Specifically IEEE 802.1Qbv, which uses time-triggered windows to prioritize critical traffic on the wire.
  • Linux Real-Time Scheduling: Assigning FIFO or Round Robin attributes to specific execution threads.
  • Pluggable Invocation Modes: The system supports three execution methods: Dynamic Library Loading (DLF), WebAssembly (WASMF), and standard process spawning (FSpawn).

Experimental Methodology

To validate TEMPOS, the authors developed a custom experimental testbed featuring heterogeneous hardware, including high-end AMD Ryzen workstations, mid-range Intel Core i5 PCs, and low-power edge boards (Intel Atom/Celeron). A specialized TSN-enabled Ethernet switch (Relyum RELY-TSN-BRIDGE) was used to manage network traffic.

Workloads were simulated using a synthetic function that performs data deserialization, string searching, and intensive mathematical operations. This allowed the researchers to measure the impact of system load on different priority levels.

Key Insights and Results

Holistic Coordination is Mandatory

The study demonstrates that improving just the network or just the CPU is insufficient for end-to-end guarantees. TEMPOS proves that synchronization must occur across the entire path—from the network wire to the message queue and finally to the CPU scheduler.

Performance Differentiation

In head-to-head tests, "Strict Quality" (SQ) traffic maintained an end-to-end latency of approximately 500µs. In contrast, "Best-effort" (BQ) traffic was significantly more volatile. Under a heavy load of 1,000 requests per second, SQ traffic remained stable at sub-millisecond levels, while BQ traffic latency spiked to over 600ms—a 1000x difference that highlights the effectiveness of the TEMPOS priority mechanisms.

Execution Environment Trade-offs

  • DLF (Dynamic Libraries): Provided the fastest, near-instant startup times.
  • WASM (WebAssembly): Offered a compelling balance of safety, portability, and performance, staying within the 1ms threshold for many tasks.
  • FSpawn: Proved the slowest due to the overhead of operating system process creation.

Limitations and Future Work

While TEMPOS successfully demonstrates QoS differentiation, the current prototype relies on static configuration files. Future iterations will require a dynamic resource orchestrator capable of reconfiguring QoS slices across a global cloud-to-edge continuum. Additionally, the authors plan to expand the middleware's scope to include QoS management for storage and hardware accelerators, such as GPUs and FPGAs, which are increasingly vital for edge-based AI tasks.

Summary updated

Abstract

Several classes of advanced Internet of Things (IoT) applications, e.g., in the industrial manufacturing domain, call for Quality of Service (QoS) management to guarantee/control performance indicators, even in presence of many sources of “stochastic noise” in real deployment environments, from scarcely available bandwidth in a time window to concurrent usage of virtualized processing resources. This paper proposes a novel IoT-oriented middleware that i) considers and coordinates together different aspects of QoS monitoring, control, and management for different kinds of virtualized resources (from networking to processing) in a holistic way, and ii) specifically targets deployment environments where edge cloud resources are employed to enable the Serverless paradigm in the cloud continuum. The reported experimental results show how it is possible to achieve the desired QoS differentiation by coordinating heterogeneous mechanisms and technologies already available in the market. This demonstrates the feasibility of effective QoS-aware management of virtualized resources in the cloud-to-things continuum when considering a Serverless provisioning scenario, which is completely original in the related literature to the best of our knowledge.
BibTeX
@inproceedings{garbugli2022tempos,
  author = {Andrea Garbugli and Andrea Sabbioni and Antonio Corradi and Paolo Bellavista},
  title = {{TEMPOS: QoS Management Middleware for Edge Cloud Computing FaaS in the Internet of Things}},
  booktitle = {IEEE Access},
  year = {2022},
  url = {https://openreview.net/forum?id=AHuAp6R6oe},
}
Figure