Abstract: A modern and flexible approach to Bayesian Optimization / Model Based Optimization building on the 'bbotk' package. 'mlr3mbo' is a toolbox providing both ready-to-use optimization algorithms as well as their fundamental building blocks allowing for straightforward implementation of custom algorithms. Single- and multi-objective optimization is supported as well as mixed continuous, categorical and conditional search spaces. Moreover, using 'mlr3mbo' for hyperparameter optimization of machine learning models within the 'mlr3' ecosystem is straightforward via 'mlr3tuning'. Examples of ready-to-use optimization algorithms include Efficient Global Optimization by Jones et al. (1998), ParEGO by Knowles (2006) and SMS-EGO by Ponweiser et al. (2008).
Introduction
Optimization is the heartbeat of modern machine learning, particularly when tuning hyperparameters or searching for optimal architectures. However, many existing Bayesian Optimization (BO) tools are "black boxes"—they work well for standard tasks but are difficult to modify for specialized research needs. The mlr3mbo package addresses the need for a framework that handles the complexities of real-world optimization, including mixed search spaces (continuous, categorical, and conditional), multi-objective trade-offs, and scalability through asynchronous execution.
A Modular Approach to Optimization
The core philosophy of mlr3mbo is modularity. It deconstructs the BO process into three primary components that users can mix and match:
- Surrogate Models: These are models of the objective function. The package provides intelligent defaults: Gaussian Processes (
default_gp) for purely numeric spaces and Random Forests (default_rf) for mixed or conditional spaces. - Acquisition Functions: These define the strategy for choosing the next point to evaluate, balancing exploration (searching unknown areas) and exploitation (refining known good areas). The package includes a wide array of choices, including Expected Improvement (EI), Confidence Bound (CB), and Expected Hypervolume Improvement (EHVI) for multi-objective tasks.
- Acquisition Optimizers: Once an acquisition function is defined, it must be optimized to find the best next step.
mlr3mbosupports various methods, from simple random searches to more complex direct search algorithms.
A standout feature is the support for Asynchronous Model-Based Optimization (AMBO). Through classes like OptimizerAsyncMbo, the framework allows multiple workers to communicate through a shared archive, enabling decentralized optimization that is significantly more efficient on modern hardware.
Key Insights and Implementation
The package utilizes "syntactic sugar" functions (such as acqf() and acqo()) to make complex object construction readable and concise, lowering the barrier to entry for complex BO setups. One of the most useful features for practitioners is the default_surrogate logic, which automatically inspects the user's search space and selects the most appropriate model, preventing common pitfalls in model selection.
mlr3mbo implements several high-impact optimization algorithms within this unified framework:
- EGO (Efficient Global Optimization): The classic sequential BO algorithm.
- ParEGO and SMS-EGO: State-of-the-art methods for multi-objective optimization.
- Constant Liar Strategy: A method for proposing multiple points at once by using placeholder results for pending evaluations to maintain diversity in the search.
Conclusion and Future Directions
mlr3mbo provides a robust infrastructure for both standard and custom optimization tasks. While it is deeply integrated into the mlr3 ecosystem—providing immense power for R users—it does involve a learning curve related to R6 classes and the specific "grammar" of the suite. Future development is focused on stabilizing the asynchronous API and expanding the library of ready-to-use surrogate models, ensuring that mlr3mbo remains a versatile tool for the evolving needs of the machine learning community.
Summary updated