LaSS: Running Latency Sensitive Serverless Computations at the Edge

Bin Wang, Ahmed Ali-Eldin, Prashant J. Shenoy

Introduction

Serverless computing, or Function-as-a-Service (FaaS), has emerged as a compelling paradigm for IoT and edge computing due to its ability to handle bursty workloads with minimal management overhead. However, the "infinite scalability" promised by cloud providers is a mirage at the network edge, where resources are finite and easily saturated. Standard FaaS platforms like Apache OpenWhisk often fail to meet Service Level Objectives (SLOs) under heavy load because they lack latency awareness and struggle with resource contention.

The Challenge: Resource Pressure at the Edge

Edge clusters are typically small, making them vulnerable to "cold starts" and cascading failures when multiple functions burst simultaneously. Existing platforms treat all functions as equal, ignoring the specific latency requirements of tasks like real-time object detection versus background data logging. When a cluster runs out of capacity, traditional systems either queue requests indefinitely or drop them, leading to significant SLO violations.

The LaSS Architecture

To address these issues, the researchers developed LaSS (Latency Sensitive Serverless). Built on top of Apache OpenWhisk, LaSS introduces a decoupled control path that centralizes decision-making, allowing for a global view of cluster health and more intelligent resource allocation.

Comparison of standard OpenWhisk architecture (a) and the proposed LaSS architecture (b).

As shown in the architectural comparison above, LaSS introduces a dedicated management component within the controller. This component interacts with a simplified invoker on the worker nodes, enabling more granular control over container lifecycles, including creation, deletion, and resizing. This separation of concerns allows the system to make global scheduling decisions that are more robust than the localized logic used in vanilla OpenWhisk.

Key Innovations: Queuing Theory and Container Deflation

LaSS employs two primary strategies to maintain low latency:

  1. Queuing-Theoretic Allocation: Instead of relying on simple resource thresholds, LaSS uses M/M/c queuing models. By analyzing request arrival rates and service times, it calculates the precise number of containers needed to ensure a target percentage of requests (e.g., 95%) finish within a specified deadline.
  2. Container Deflation: This is the system's most novel feature. When the cluster is overloaded, rather than terminating containers to free up resources—which would trigger expensive cold starts later—LaSS "deflates" them. By dynamically reducing the CPU cycles allocated to a running container via OS-level system calls, the system keeps functions alive. While a deflated container processes requests more slowly, it prevents the massive latency spikes associated with queuing or restarting functions.

Experimental Evaluation

The effectiveness of LaSS was evaluated using a diverse set of edge-centric workloads and realistic invocation patterns derived from the 2019 Azure Functions trace.

Summary of the benchmark functions used to evaluate LaSS, including their resource requirements and implementation languages.

The benchmark suite (detailed in the table above) includes compute-intensive Deep Neural Network (DNN) models like MobileNet v2 and SqueezeNet, alongside utility functions like Image Resizer and GeoFencing. This variety ensures that the system's performance is tested across different resource profiles and programming environments.

Key Results and Insights

The evaluation revealed several critical insights:

  • Concurrency over Speed: A major finding was that maintaining a higher number of "slow" (deflated) containers is often preferable to having fewer "fast" ones. This approach maximizes concurrency and minimizes the time requests spend waiting in queues.
  • Efficiency and Stability: LaSS improved overall system utilization by 6% compared to termination-based policies. More importantly, it remained stable in "over-packed" scenarios where vanilla OpenWhisk would typically crash.
  • Rapid Response: The system can react to a 10% load spike in under 100ms, making it highly suitable for the volatile nature of edge environments.

Future Directions

While LaSS significantly improves edge serverless performance, challenges remain. Currently, the system only deflates CPU resources; memory deflation is more complex due to the risk of Out-of-Memory (OOM) kills. Future work will also look into managing resources for complex function chains and generalizing the queuing models beyond Poisson distributions to better capture diverse traffic patterns.

Summary updated

BibTeX
@inproceedings{wang2021lass,
  author = {Bin Wang and Ahmed Ali-Eldin and Prashant J. Shenoy},
  title = {{LaSS: Running Latency Sensitive Serverless Computations at the Edge}},
  booktitle = {HPDC},
  year = {2021},
  doi = {10.1145/3431379.3460646},
}
Figure