Abstract
The goal of this article is to provide an introduction to the desirability function approach to multi-objective optimization (direct and surrogate model-based), and multi-objective hyperparameter tuning. This work is based on the paper by Kuhn (2016). It presents a Python implementation of Kuhn's R package desirability. The Python package spotdesirability is available as part of the sequential parameter optimization framework. After a brief introduction to the desirability function approach is presented, three examples are given that demonstrate how to use the desirability functions for classical optimization, surrogate-model based optimization, and hyperparameter tuning.
Introduction
In real-world engineering and machine learning, optimization problems rarely involve a single objective. A chemical engineer might seek to maximize product yield while strictly maintaining safety temperatures; a data scientist aims to maximize model accuracy while minimizing computational training time. The fundamental challenge lies in the fact that these objectives exist on disparate scales—percentages versus seconds, for instance.
Traditional approaches often rely on weighted sums to aggregate multiple objectives. However, selecting these weights is frequently arbitrary and lacks intuitive physical meaning. The desirability function approach addresses this by mapping diverse metrics onto a unified, interpretable [0, 1] scale. While the R programming language has long possessed robust tools for this methodology, the Python ecosystem lacked a comparable implementation until now. This paper introduces spotdesirability, a Python package that fills this gap, enabling complex trade-off management in classical optimization, surrogate modeling, and hyperparameter tuning (HPT).
The Desirability Function Approach
The core methodology, originally proposed by Derringer and Suich, follows a three-step process:
- Transformation: Each objective is mapped to an individual desirability score ($d_r$) between 0 (completely unacceptable) and 1 (ideal). The
spotdesirabilitypackage supports several transformation types:DMax(larger-is-better),DMin(smaller-is-better), andDTarget(target-is-best). The shape of these functions can be tuned using a scale parameter to reflect the relative importance of approaching the ideal value.
- Aggregation: Individual scores are combined into a single overall desirability index ($D$) using the geometric mean. This choice is critical: if any single objective is deemed "unacceptable" (a score of 0), the overall desirability becomes 0, ensuring that all critical constraints are respected.
- Optimization: The resulting desirability index can then be optimized using standard techniques. The paper demonstrates the use of
scipy.optimizefor classical problems and thespotoptimpackage for surrogate-model-based optimization using Gaussian Processes.
Additionally, the author introduces an "intensive" version of the Morris-Mitchell criterion for experimental design. By normalizing the criterion by the total number of point pairs, the metric becomes size-invariant, allowing for fair comparisons of space-filling quality across designs of different sample sizes.
Applications and Validation
The spotdesirability package was validated across three distinct scenarios:
- Chemical Reaction Optimization: Using Response Surface Methodology (RSM) to optimize reaction time, temperature, and catalyst percentage for conversion and activity.
- Hyperparameter Tuning (HPT): Tuning a PyTorch neural network on the scikit-learn diabetes dataset, balancing validation loss against training epochs. The optimizer successfully identified a configuration with a high desirability score (0.9588).
- Mathematical Benchmarking: Testing the exploration-exploitation trade-off using the Ackley function.
A key innovation presented is the use of Infill-Diagnostic Plots. By visualizing the distribution of newly suggested "infill points" against existing data, practitioners can monitor whether the optimizer is effectively exploring the search space or converging prematurely. Furthermore, the paper demonstrates a "two-phase" optimization strategy: initially combining performance desirability with space-fillingness to explore the domain, then transitioning to pure performance desirability to refine the final solution.
Discussion and Future Directions
The desirability function approach offers superior interpretability compared to arbitrary weighting. Desirability limits (e.g., "yield below 80% is unacceptable") have direct physical or business relevance. However, simple power-based desirability functions can sometimes create "plateaus"—flat regions in the search space where optimizers may struggle due to a lack of gradient.
Future work will focus on developing more sophisticated, non-linear desirability functions—such as the logistic function shown above—to mitigate these plateaus. Furthermore, while initial HPT results are promising, extensive benchmarking against established methods like Bayesian Optimization or Hyperband is necessary to fully characterize the performance of desirability-based tuning.
Conclusion
The spotdesirability package successfully brings a powerful multi-objective optimization framework to the Python community, achieving parity with established R tools. By providing a structured way to handle trade-offs and visualize optimization progress, it offers a robust solution for complex engineering and machine learning challenges.
Summary updated