Optimizing Simultaneous Autoscaling for Serverless Cloud Computing

Chen Wang, Harold J. Ship, Evgeny Shindin, Chen Wang 0039, Diana Arroyo, Asser N. TantawiCloud 2023

Abstract: This paper explores resource allocation in serverless cloud computing platforms and proposes an optimization approach for autoscaling systems. Serverless computing relieves users from resource management tasks, enabling focus on application functions. However, dynamic resource allocation and function replication based on changing loads remain crucial. Typically, autoscalers in these platforms utilize threshold-based mechanisms to adjust function replicas independently. We model applications as interconnected graphs of functions, where requests probabilistically traverse the graph, triggering associated function execution. Our objective is to develop a control policy that optimally allocates resources on servers, minimizing failed requests and response time in reaction to load changes. Using a fluid approximation model and Separated Continuous Linear Programming (SCLP), we derive an optimal control policy that determines the number of resources per replica and the required number of replicas over time. We evaluate our approach using a simulation framework built with Python and simpy. Comparing against threshold-based autoscaling, our approach demonstrates significant improvements in average response times and failed requests, ranging from 15% to over 300% in most cases. We also explore the impact of system and workload parameters on performance, providing insights into the behavior of our optimization approach under different conditions. Overall, our study contributes to advancing resource allocation strategies, enhancing efficiency and reliability in serverless cloud computing platforms.

Introduction

Serverless computing has revolutionized cloud application development by abstracting away infrastructure management. However, the underlying challenge of dynamic resource allocation remains. Current platforms typically employ reactive, threshold-based autoscalers that operate on individual functions in isolation. This approach often fails to account for the interdependencies in modern microservice "chains," leading to bottlenecks, cold starts, and resource inefficiency. This paper proposes a proactive, system-wide optimization strategy to address these limitations by moving from reactive "if-then" rules to a proactive global optimization.

Methodology and System Architecture

The researchers model serverless applications as interconnected graphs of functions within a Multi-class Queueing Network (MCQN). To handle the scale of modern cloud environments, they utilize a fluid approximation model, treating discrete request arrivals as a continuous flow. This allows the use of powerful linear programming tools that can solve massive problems involving hundreds of functions in seconds.

System architecture of the serverless environment showing functions, load balancers, and resource replicas across multiple servers.

The core of the proposed solution is the application of Separated Continuous Linear Programming (SCLP). By solving the SCLP problem using a specialized "Revised SCLP-Simplex" algorithm, the system derives an optimal control policy. This policy simultaneously determines both the number of replicas and the specific resource allocation (e.g., CPU units) per replica over a defined time horizon, anticipating the needs of downstream functions in a chain.

Performance Evaluation

The approach was validated using a custom simulation framework built with Python and simpy. The evaluation compared the "fluid policy" against standard threshold-based autoscaling across various network topologies, including a "criss-cross" model and large-scale networks with up to 500 function types across 100 servers.

Comparison of cumulative arrivals and departures for various services under standard autoscaling versus the proposed fluid optimization approach.

Key findings from the simulation include:

  • Response Time Improvements: The fluid policy achieved approximately 75% improvement in average response times across many network size tests.
  • Reduction in Failed Requests: In highly heterogeneous workloads where functions have wildly different speeds, the standard autoscaler exhibited over 300% more failures compared to the fluid optimization approach.
  • Scalability: Performance remained consistent as the system scaled from 50 to 500 function types, proving the model scales well with system size.
  • Resource Efficiency: The fluid model utilized server capacity more fully by being flexible with replica counts, whereas threshold-based models often left resources idle.

Conclusion and Future Work

By moving from reactive rules to proactive global optimization, this research demonstrates significant gains in serverless efficiency and reliability. While the fluid model shows some sensitivity to stochastic jitter in very small queues, its ability to handle complex function chains and heterogeneous loads makes it a promising candidate for next-generation serverless platforms. Future work will focus on real-world implementation in environments like Knative or OpenLambda to account for physical overheads such as network latency and container startup times.

Summary updated

Abstract

This paper explores resource allocation in serverless cloud computing platforms and proposes an optimization approach for autoscaling systems. Serverless computing relieves users from resource management tasks, enabling focus on application functions. However, dynamic resource allocation and function replication based on changing loads remain crucial. Typically, autoscalers in these platforms utilize threshold-based mechanisms to adjust function replicas independently. We model applications as interconnected graphs of functions, where requests probabilistically traverse the graph, triggering associated function execution. Our objective is to develop a control policy that optimally allocates resources on servers, minimizing failed requests and response time in reaction to load changes. Using a fluid approximation model and Separated Continuous Linear Programming (SCLP), we derive an optimal control policy that determines the number of resources per replica and the required number of replicas over time. We evaluate our approach using a simulation framework built with Python and simpy. Comparing against threshold-based autoscaling, our approach demonstrates significant improvements in average response times and failed requests, ranging from 15% to over 300% in most cases. We also explore the impact of system and workload parameters on performance, providing insights into the behavior of our optimization approach under different conditions. Overall, our study contributes to advancing resource allocation strategies, enhancing efficiency and reliability in serverless cloud computing platforms.
BibTeX
@inproceedings{wang2023optimizing,
  author = {Chen Wang and Harold J. Ship and Evgeny Shindin and Chen Wang and Diana Arroyo and Asser N. Tantawi},
  title = {{Optimizing Simultaneous Autoscaling for Serverless Cloud Computing}},
  booktitle = {Cloud},
  year = {2023},
}
Figure