MALT: Fine-Grained Microservice Profiling for Request Latency Anomaly Localization

Dingjie Zhang, Qinghua Wu, Yuankang Zhao, Zhenyu Li, Peng He, Yongsu Zhang, Xiongchun Duan, Yu Zhang, Gen Miao, Jiancong Tong, Gaogang Xie

Abstract

Recent years have witnessed the shift of cloud datacenter architecture from monolithic to microservice. In microservice, request latency which fundamentally determines application performance, has become a crucial concern for cloud operators. Due to various issues with hosts, networks, and remote services, microservices can encounter anomalies when processing requests, causing sudden latency spikes and potential Service- Level Agreement (SLA) violations. Existing performance monitoring tools lack the ability of collecting performance data containing request-level semantics under low overhead. They also do not address the diverse threading models of microservices, posing challenges in locating Request Latency Anomalies (RLAs). This paper proposes MALT, a performance profiling tool that collects fine-grained, request-level data and locates RLAs in microservices. MALT hooks the critical functions when microser-vices process requests and extracts request-level semantic data from function parameters or packet payloads. To characterize the complex and diverse microservice threading behaviors, MALT maps the collected data to each request according to request-level semantics and models the request execution process of microservices as a Directed Acyclic Graph (DAG), from which the path with longest latency is extracted to locate RLAs. We apply MALT to an open-source microservice in an experimental network and a microservice deployed in a production network. Results show that MALT achieves an average precision of 93% in locating four types of RLAs. These results further guide the optimization of request latency in production microservice, where the average request latency is reduced by up to 41.8 % by the heuristic methods.
BibTeX
@inproceedings{zhang2023malt,
  author = {Dingjie Zhang and Qinghua Wu and Yuankang Zhao and Zhenyu Li and Peng He and Yongsu Zhang and Xiongchun Duan and Yu Zhang and Gen Miao and Jiancong Tong and Gaogang Xie},
  title = {{MALT: Fine-Grained Microservice Profiling for Request Latency Anomaly Localization}},
  booktitle = {HPCC/DSS/SmartCity/DependSys},
  year = {2023},
  doi = {10.1109/hpcc-dss-smartcity-dependsys60770.2023.00025},
}

Loading related content...

Figure