Abstract
The accuracy of some density functional (DF) models widely used in material science depends on empirical or free parameters that are commonly tuned using reference physical properties. Grid-search methods are the standard numerical approximations used to find the optimal values of the free parameters, making the computational complexity to scale with the number of points in the grid. In this report, we illustrate that Bayesian optimization (BO), a sample-efficient machine learning algorithm, can calibrate different density functional models, e.g., hybrid exchange-correlation and range-separated density functionals. Using the atomization energies and bond lengths from the Gaussian-1 (G1) and Gaussian-2 (G2) databases, we show that BO optimizes the free parameters of the hybrid exchange-correlation functionals with approximately 55 evaluations of the error function. We also show that selecting exchange-correlation functionals for different physical systems can be done with BO. We jointly optimize and select the form of the exchange-correlation functionals and the free parameters by minimizing the root-mean-square error functions for the G1 and G2 data set atomization energies. The calibrated DF model is more accurate on average than standard DF methods, e.g., PBE0 and B3LYP.
Introduction
Density Functional Theory (DFT) serves as the primary tool for predicting molecular and material properties. However, many widely used hybrid functionals incorporate empirical parameters that must be calibrated against experimental data. Traditionally, this calibration relies on grid-search methods, which become computationally prohibitive as the number of parameters increases. Furthermore, selecting the optimal exchange-correlation (XC) functional from the vast "zoo" of available options often requires exhaustive manual benchmarking.
Methodology: Bayesian Optimization
To address these challenges, we employ Bayesian Optimization (BO), a machine learning strategy optimized for expensive "black-box" functions. BO utilizes a Gaussian Process (GP) as a surrogate model to map the error landscape, providing both expected values and uncertainty estimates.
The algorithm navigates this landscape using acquisition functions like Expected Improvement (EI) and Upper Confidence Bound (UCB). These functions balance the exploration of uncertain regions with the exploitation of known low-error areas. As shown in Figure 1, BO significantly outperforms random or grid-based sampling, reaching optimal parameter values in roughly 55 evaluations.
Joint Optimization and Model Selection
A key innovation in this work is the joint optimization of both the functional form and its internal parameters. By mapping discrete functional combinations to a continuous space using integer IDs and floor functions, BO can simultaneously search across 30 different XC combinations.
This approach automates model selection, identifying the most physically appropriate correlation functional for a given system without manual intervention. The heatmap in Figure 4 illustrates the diverse error landscape across different exchange and correlation pairings, which BO efficiently traverses to find the global minimum.
Key Results
- Enhanced Accuracy: For the PBE0 functional, BO identified an optimal exchange parameter ($a_0 = 0.1502$) that reduced the Root Mean Square Error (RMSE) to 10.08 kcal/mol, a notable improvement over the standard PBE0 error of 11.30 kcal/mol.
- Superior Functionals: BO discovered a version of the B-LYP functional with an RMSE of 7.91 kcal/mol, outperforming the ubiquitous B3LYP (9.48 kcal/mol).
- Structural Precision: In bond length optimization, BO-tuned models achieved an RMSE of ~0.007 Å, surpassing standard PBE0 (~0.011 Å).
- Basis Set Sensitivity: The study confirms that optimal parameters are sensitive to the choice of basis set. BO provides a systematic framework for re-calibrating functionals whenever the computational setup changes.
Conclusion
Bayesian Optimization offers a sample-efficient alternative to traditional grid searches for DFT calibration. By automating both parameter tuning and functional selection, BO enables the development of "custom" functionals that are more accurate for specific datasets or properties than standard off-the-shelf models. Future work will focus on validating these findings on larger datasets like G3 and exploring more complex functional spaces, including meta-GGAs and range-separated models.
Summary updated