Log 2 : a cost-aware logging mechanism for performance diagnosis

Rui Ding, Hucheng Zhou, Jian–Guang Lou, Hongyu Zhang, Qingwei Lin, Qiang Fu, Dongmei Zhang, Tao Xie

Abstract

Logging has been a common practice for monitoring and diagnosing performance issues. However, logging comes at a cost, especially for large-scale online service systems. First, the overhead incurred by intensive logging is non-negligible. Second, it is costly to diagnose a performance issue if there are a tremendous amount of redundant logs. Therefore, we believe that it is important to limit the overhead incurred by logging, without sacrificing the logging effectiveness. In this paper we propose <i>Log</i>², a cost-aware logging mechanism. Given a “budget” (defined as the maximum volume of logs allowed to be output in a time interval), <i>Log</i>² makes the “whether to log” decision through a two-phase filtering mechanism. In the first phase, a large number of irrelevant logs are discarded efficiently. In the second phase, useful logs are cached and output while complying with logging budget. In this way, <i>Log</i>² keeps the useful logs and discards the less useful ones. We have implemented <i>Log</i>² and evaluated it on an open source system as well as a real-world online service system from Microsoft. The experimental results show that <i>Log</i>² can control logging overhead while preserving logging effectiveness.
BibTeX
@inproceedings{ding2015log,
  author = {Rui Ding and Hucheng Zhou and Jian–Guang Lou and Hongyu Zhang and Qingwei Lin and Qiang Fu and Dongmei Zhang and Tao Xie},
  title = {{Log 2 : a cost-aware logging mechanism for performance diagnosis}},
  booktitle = {USENIX ATC},
  year = {2015},
  url = {https://openreview.net/forum?id=oRbQDQdwGA},
}

Loading related content...

Figure