ChainsFormer: A Chain Latency-Aware Resource Provisioning Approach for Microservices Cluster

Chenghao Song, Minxian Xu, Kejiang Ye, Huaming Wu, Sukhpal Singh Gill, Rajkumar Buyya, Chengzhong XuLecture Notes in Computer Science 2023 · DOI · OpenAlex

Abstract

The shift from monolithic applications to microservices has created lightweight, fine-grained components linked via complex inter-dependencies. Managing these clusters requires provisioning optimal resources to ensure performance while maintaining efficiency. ChainsFormer is a framework that analyzes microservice inter-dependencies to identify critical chains and nodes, utilizing reinforcement learning for resource provisioning. By combining vertical and horizontal scaling, ChainsFormer reduces response time by up to 26% and improves throughput by 8% compared to state-of-the-art techniques.

Introduction: The Challenge of Microservice Chains

Modern cloud-native architectures decompose monolithic applications into dozens of inter-dependent microservices. While this modularity enhances agility, it complicates resource management. A single user request often traverses a "call chain" of multiple services; under-provisioning a single node in this chain can cause a bottleneck that degrades the entire application's performance.

Existing resource management tools, such as Kubernetes Horizontal Pod Autoscaler (HPA), typically monitor metrics like CPU usage in isolation. They often fail to account for the "ripple effect" where latency in one service propagates through the dependency graph. Conversely, advanced research models that use heavy machine learning or centralized graph databases often struggle with the highly dynamic nature of microservice clusters.

Microservice dependency graph and CPU utilization heatmap illustrating the propagation of resource demands across a service chain.

Methodology: The ChainsFormer Framework

ChainsFormer addresses these challenges through a three-tiered architecture designed for proactive and chain-aware management.

1. Neural Network-based Workload Predictor

To move from reactive to proactive scaling, ChainsFormer employs a Gated Recurrent Unit (GRU) model named esDNN. This model forecasts future traffic patterns, allowing the system to prepare resources before a spike occurs.

2. Chains Analyzer

The "brain" of the framework is the Chains Analyzer. It identifies the critical chain—the path with the highest end-to-end latency—using a weighted longest-path algorithm. Once the chain is identified, a lightweight decision tree pinpoints the specific critical nodes causing the bottleneck. This approach is more responsive to transient performance dips than heavier graph-based models.

3. RL-based Resource Scaler

For resource allocation, ChainsFormer uses a Reinforcement Learning (RL) agent based on the SARSA algorithm. Unlike standard Q-Learning, SARSA updates its strategy based on the actual policy being followed, leading to faster convergence in dynamic environments. The agent performs hybrid scaling, choosing between adding replicas (horizontal scaling) or adjusting the CPU/memory of existing containers (vertical scaling) to resolve bottlenecks efficiently.

Evaluation and Datasets

The effectiveness of ChainsFormer was evaluated on a physical 5-node Kubernetes cluster using realistic benchmarks:

  • Alibaba Cluster Trace: A real-world dataset containing 8 days of traces from over 4,000 servers, used to simulate complex workload patterns.
  • Microservice Benchmarks: The Sock Shop and Train-Ticket applications were used to represent varying levels of architectural complexity.
  • Traffic Generation: Realistic traffic patterns were generated using Locust based on the Alibaba traces.
Workload patterns from Alibaba traces showing CPU usage and discretized utilization levels over days and minutes.

Key Results

Experimental results demonstrate that ChainsFormer significantly outperforms native Kubernetes (KS), Google Autopilot-inspired (AUTO), and the state-of-the-art FIRM framework.

  • Latency and Throughput: ChainsFormer reduced average response times by 1.4% to 26.6% and improved processed requests per second (RPS) by up to 8.1%.
  • Reliability: In long-term scenarios with high workload variance, ChainsFormer reduced failed requests by 8.3% compared to FIRM.
  • Scalability: During stress tests (increasing load from 400 to 1200 RPS), ChainsFormer maintained a flatter performance profile, handling spikes more gracefully than competing methods.

Conclusion

ChainsFormer demonstrates that "chain-awareness" is essential for efficient microservice management. By focusing on critical paths and utilizing a hybrid scaling approach driven by reinforcement learning, it achieves superior performance and reliability. Future work will explore optimizing the RL reward function to incorporate energy consumption and cost-efficiency for production environments.

Summary updated

BibTeX
@inproceedings{song2023chainsformer,
  author = {Chenghao Song and Minxian Xu and Kejiang Ye and Huaming Wu and Sukhpal Singh Gill and Rajkumar Buyya and Chengzhong Xu and Cheng-zhong Xu},
  title = {{ChainsFormer: A Chain Latency-Aware Resource Provisioning Approach for Microservices Cluster}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2023},
  doi = {10.1007/978-3-031-48421-6\_14},
}

Loading related content...

Figure