Abstract
This paper introduces NOWPAC (Nonlinear Optimization With Path-Augmented Constraints), a derivative-free optimization (DFO) algorithm designed for complex engineering problems where objective and constraint functions are "black boxes." NOWPAC utilizes a trust-region framework with fully linear models and a novel inner boundary path (IBP) scheme to ensure feasible trial steps. We provide a rigorous proof of convergence to first-order critical points and introduce a noise indicator to handle numerical errors in simulation data.
Introduction: The Challenge of Black-Box Optimization
In modern engineering, performance evaluation often relies on high-fidelity simulations. These simulations are typically "black boxes": they provide outputs $f(x)$ and constraint values $c(x)$ for given inputs $x$, but offer no gradient information. This presents two significant hurdles:
- Lack of Derivatives: Standard gradient-based optimization cannot be applied directly.
- Numerical Noise: Simulations often contain small numerical errors. As an optimizer approaches a solution, these "wiggles" can be misinterpreted as local optima, causing the algorithm to fail.
While various DFO methods exist, few provide the mathematical guarantees of convergence required for reliable industrial application, especially under general nonlinear constraints.
Methodology: The NOWPAC Framework
NOWPAC operates using a trust-region framework. At each iteration, it constructs local surrogate models (linear or quadratic approximations) of the objective and constraints based on sampled simulation data.
The Inner Boundary Path (IBP)
The defining feature of NOWPAC is the Inner Boundary Path (IBP). Rather than simply attempting to satisfy constraints, NOWPAC introduces an adaptive offset that pushes the search into the interior of the feasible set.
This mechanism serves two purposes:
- Local Convexification: It smooths the feasible region, facilitating the identification of valid search directions.
- Feasibility Buffer: By maintaining a distance from the boundaries, it ensures that trial steps remain "safe," preventing simulation crashes caused by infeasible inputs.
Noise Detection via Hessian Monitoring
To address numerical noise, NOWPAC monitors the curvature (Hessian) of its surrogate models. An unnatural spike in the Hessian norm relative to the trust-region size indicates that the algorithm is "fitting the noise." This trigger allows the solver to terminate or adjust its precision, preventing it from wasting computational resources on spurious data features.
Experimental Evaluation
The authors validated NOWPAC across a diverse set of problems:
- Analytical Benchmarks: Testing on the Rosenbrock function and constrained anisotropic exponential problems.
- Schittkowski Library: A comprehensive suite of over 300 nonlinear programming problems.
- Industrial Case Study: A simulation of a tar removal process in a biomass-to-liquid plant, characterized by expensive evaluations and inherent numerical noise.
Key Results
- Superior Efficiency: On the Schittkowski benchmarks, NOWPAC required significantly fewer function evaluations than established solvers like COBYLA, NOMAD, and GSS-NLC.
- Strict Feasibility: In the industrial tar removal study, NOWPAC identified more efficient designs than COBYLA while strictly adhering to safety constraints—a task where COBYLA occasionally failed.
- Robustness: The noise indicator successfully prevented the algorithm from "chasing ghosts" in noisy environments, ensuring reliable termination at the limits of simulation precision.
Conclusion and Future Work
NOWPAC offers a mathematically rigorous and computationally efficient solution for constrained black-box optimization. By combining an innovative IBP scheme with robust noise detection, it bridges the gap between theoretical convergence and practical engineering needs. Future research will focus on extending NOWPAC to global optimization and stochastic settings where objective functions are subject to random variation.
Summary updated