Centrum: Model-based Database Auto-tuning with Minimal Distributional Assumptions

Yuanhao Lai, Pengfei Zheng, C. S. Ji, Yan Li, Songhan Zhang, Rutao Zhang, Zhengang Wang, Yunfei Du

TL;DR

This paper introduces "Centrum," a novel database auto-tuning system that significantly improves performance by using a gradient boosting ensemble model with minimal distributional assumptions. Centrum achieves state-of-the-art results by combining advanced surrogate modeling with distribution-free uncertainty estimation, outperforming existing methods in both tuning effectiveness and efficiency.

What Problem Does This Paper Address?

Database Management Systems (DBMS) are complex and require careful configuration to perform optimally across diverse workloads and hardware. Traditional manual tuning by Database Administrators (DBAs) is time-consuming, error-prone, and doesn't scale. While Bayesian Optimization (BO) has emerged as a powerful automated approach, existing BO-based auto-tuners often rely on Gaussian Process (GP) models. These GPs make strong assumptions about the data distribution (e.g., continuity, Gaussianity, homoscedasticity) that are frequently violated in real-world DBMS performance data. This violation leads to inaccurate performance predictions and suboptimal tuning. Even tree-ensemble methods, while better, still struggle with accurate uncertainty estimation. The paper aims to bridge this gap by developing an auto-tuner that is robust to these distributional assumptions, leading to more effective and efficient DBMS tuning.

Methods

Centrum is a model-based auto-tuner that leverages a novel combination of techniques:

  • Stochastic Gradient Boosting Ensembles (SGBE): Centrum uses SGBE as its core surrogate model. This ensemble approach, based on gradient boosting, is designed to provide accurate point estimates of DBMS performance without making strong distributional assumptions.
  • Two-Phase Learning: A key innovation is a two-phase learning procedure for SGBE. The first phase trains a conformalized SGBE, and the second phase fine-tunes the ensemble to co-optimize both point and interval (uncertainty) estimation accuracy.
  • Locally Adaptive Conformalized Ensemble: To achieve distribution-free interval estimation, Centrum employs a generalized locally adaptive conformal method. This technique, based on conformal inference, provides robust uncertainty estimates that adapt to local input characteristics, addressing limitations of constant-width intervals.
  • Distribution-Free Acquisition Function: Centrum constructs its acquisition function (which guides the search for the next configuration to test) using these distribution-free interval estimates, ensuring that exploration is not distorted by incorrect uncertainty assumptions.

Datasets

The paper uses a combination of physical and simulated experiments for evaluation.

  • Physical Experiments: Conducted on three DBMSs (MySQL-v8.0, MySQL-v5.7, PostgreSQL-v10.5) with three workloads (JOB, TPCC, Sysbench).
  • Simulation Experiments: Utilized an open benchmark [68] that provides ML-based simulators (random forest models) for DBMS performance. To mitigate potential bias from tree-ensemble simulators, the authors re-trained these simulators using transformer-based models (specifically, SAINT) for a more robust evaluation.

Key Insights and Ideas

  • Distributional Assumptions are a Bottleneck: The paper strongly argues that the fundamental distributional assumptions of GP-based BO are a major limitation in DBMS auto-tuning, leading to poor performance.
  • Tree Ensembles are Better, But Not Perfect: While tree-ensemble models (like Random Forests and Gradient Boosting) are more robust to distributional assumptions for point estimates, they still struggle with accurate and distribution-free interval estimation.
  • Conformal Inference is Key for Uncertainty: Centrum's success hinges on using conformal inference to provide distribution-free, locally adaptive uncertainty estimates, which are crucial for effective exploration in BO.
  • Co-optimization of Point and Interval Estimates: Simply improving point estimates isn't enough. Centrum's two-phase fine-tuning process explicitly co-optimizes both point and interval accuracy, leading to a more balanced and effective surrogate model.
  • Tuning Efficiency Matters: Beyond just finding a good configuration, the speed at which an optimizer finds it is critical. Centrum demonstrates significant improvements in tuning efficiency.

Key Results

  • Superior Performance: Centrum outperforms 21 state-of-the-art (SOTA) DBMS auto-tuners, including GP-BOs, tree-ensemble BOs, and other advanced methods, achieving an average of 19.2% and 29.0% higher tuned throughput or lower latency in physical and simulation experiments, respectively.
  • Improved Surrogate Accuracy: Centrum's surrogate models show significantly higher SR² (point-estimate accuracy) and NAIS (interval-estimate accuracy) compared to other methods. For instance, it achieves 92.6% higher SR² and 105.7% higher NAIS than GP-BO counterparts.
  • Enhanced Tuning Efficiency: Centrum achieves a 4.2x speedup compared to existing methods, meaning it finds optimal configurations much faster.
  • Outperforms OOBC Methods: Centrum shows a notable improvement over existing Out-of-Bag-Conformalized (OOBC) methods, demonstrating the effectiveness of its generalized locally adaptive conformal approach.
  • Ablation Study Highlights Contributions: The ablation study reveals that conformal inference contributes about half of Centrum's improvement, while the co-training and SGBE base surrogate model also play significant roles.

Limitations and Future Work

  • Computational Cost: While Centrum is more efficient in tuning, the training of SGBE and conformal ensembles can still be computationally intensive.
  • Generalizability of Conformal Methods: The paper mentions that existing Error Re-weighted Conformal (ERC) methods can have instability issues due to difficulty estimation. While Centrum proposes improvements, further research into robust conformal methods might be beneficial.
  • Future Work: The paper suggests that the core components of Centrum (SGBE, conformal inference) could be applied to other complex optimization problems beyond DBMS tuning. Further exploration of knob selection and knowledge transfer mechanisms within the Centrum framework could also be valuable.

Summary updated

Abstract

Gaussian Process (GP)-based Bayesian optimization (BO), i.e., GP-BO, emerges as a prevailing model-based framework for DBMS (Database Management System) auto-tuning. However, recent work shows GP-BO-based DBMS auto-tuners are significantly outperformed by auto-tuners based on SMAC, which features random forest surrogate models; such results motivate us to rethink and investigate the limitations of GP-BO in auto-tuner design. We find that the fundamental assumptions of GP-BO are widely violated when modeling and optimizing DBMS performance, while tree-ensemble-BOs (e.g., SMAC) can avoid the assumption pitfalls and deliver improved tuning efficiency and effectiveness. Moreover, we argue that existing tree-ensemble-BOs restrict further advancement in DBMS auto-tuning. First, existing tree-ensemble-BOs can only achieve distribution-free point estimates, but still impose unrealistic distributional assumptions on uncertainty (interval) estimates, which can compromise surrogate modeling and distort the acquisition function. Second, recent advances in (ensemble) gradient boosting, which can further enhance surrogate modeling against vanilla GP and random forest counterparts, have rarely been applied in optimizing DBMS auto-tuners. To address these issues, we propose a novel model-based DBMS auto-tuner, Centrum . Centrum achieves and improves distribution-free point and interval estimation in surrogate modeling with a two-phase learning procedure of stochastic gradient boosting ensembles (SGBE). Moreover, Centrum adopts a generalized SGBE-estimated locally-adaptive conformal prediction to facilitate a distribution-free interval (uncertainty) estimation and acquisition function. To our knowledge, Centrum is the first auto-tuner that realizes distribution-freeness to stress and enhance BO's practicality in DBMS auto-tuning, and the first to seamlessly fuse gradient boosting ensembles and conformal inference in BO. Extensive physical and simulation experiments on two DBMSs and three workloads show that Centrum outperforms 21 state-of-the-art (SOTA) DBMS auto-tuners based on BO with GP, random forest, gradient boosting, OOB (Out-Of-Bag) conformal ensemble and other surrogates, as well as that based on reinforcement learning and genetic algorithms.
BibTeX
@inproceedings{lai2025centrum,
  author = {Yuanhao Lai and Pengfei Zheng and C. S. Ji and Yan Li and Songhan Zhang and Rutao Zhang and Zhengang Wang and Yunfei Du},
  title = {{Centrum: Model-based Database Auto-tuning with Minimal Distributional Assumptions}},
  booktitle = {Proceedings of the ACM on Management of Data},
  year = {2025},
  doi = {10.1145/3709671},
}
Figure