Abstract
Optimization problems with uncertain black-box constraints, modeled by warped Gaussian processes, have recently been considered in the Bayesian optimization setting. This work introduces a new class of constraints in which the same black-box function occurs multiple times evaluated at different domain points. Such constraints are important in applications where, e.g., safety-critical measures are aggregated over multiple time periods. Our approach, which uses robust optimization, reformulates these uncertain constraints into deterministic constraints guaranteed to be satisfied with a specified probability, i.e., deterministic approximations to a chance constraint. This approach extends robust optimization methods from parametric uncertainty to uncertain functions modeled by warped Gaussian processes. We analyze convexity conditions and propose a custom global optimization strategy for non-convex cases. A case study derived from production planning and an industrially relevant example from oil well drilling show that the approach effectively mitigates uncertainty in the learned curves. For the drill scheduling example, we develop a custom strategy for globally optimizing integer decisions.
Introduction
In many engineering and economic domains, decision-making relies on "black-box" functions that are both expensive to evaluate and inherently uncertain. Standard Gaussian Processes (GPs) are frequently used to model these functions; however, they typically assume symmetric, Gaussian noise distributions. In practice, data is often skewed or exhibits heteroscedasticity (non-uniform noise).
This paper introduces a robust optimization framework designed for constraints modeled by warped Gaussian Processes. Warped GPs map observations into a latent space where they behave as standard Gaussians, allowing for more accurate modeling of complex data. The authors specifically address constraints that aggregate multiple evaluations of the same function—such as cumulative equipment wear—ensuring that safety thresholds are met with high probability without excessive conservatism.
Theoretical Framework and Methodology
The proposed methodology bridges the gap between warped GPs and Robust Optimization through several key innovations:
- Latent Space Ellipsoids: The framework constructs confidence ellipsoids in the latent space of the warped GP. These ellipsoids contain the true function values with a specified probability (e.g., 95%).
- Deterministic Reformulation: By applying Wolfe duality, the authors transform the robust (worst-case) constraints into a set of deterministic equations. This allows the use of standard, efficient solvers like Ipopt, avoiding the computational burden of Monte Carlo simulations.
- Convexity and Global Search: The paper identifies conditions under which the warping function maintains convexity. For non-convex scenarios, a spatial branching (divide-and-conquer) algorithm is employed to ensure global optimality.
- Iterative Refinement: To mitigate potential over-conservatism, an "iterative a posteriori" procedure adjusts the uncertainty set size based on simulated performance.
Case Studies and Results
The framework was validated using two distinct case studies:
Production Planning
Using a synthetic price-supply curve with non-uniform Gaussian noise, the authors demonstrated that the robust approach consistently met target feasibility. The iterative procedure successfully tightened the "a priori" bounds to match desired confidence levels, proving the method's adaptability to different noise profiles.
Drill Scheduling
In an industrially relevant oil well drilling model, the framework handled mud motor degradation and bit-rock interaction. The model intelligently "hedged" against noise by slowing drilling speeds when uncertainty was high, significantly reducing the risk of motor failure. Furthermore, a custom boundary heuristic for integer decisions (e.g., maintenance stops) matched the performance of exhaustive searches while operating significantly faster.
Key Insights and Conclusion
This research highlights the power of combining warped GPs with robust optimization for aggregated safety measures. By leveraging Wolfe duality, the approach provides a computationally efficient "shortcut" to safety guarantees in complex, non-Gaussian environments.
While the current method can become conservative as the number of evaluation points ($T$) increases, it represents a significant step forward in handling cumulative constraints. Future research will focus on more compact reformulations to reduce the number of variables and scaling the technique for larger datasets where GP training becomes a bottleneck.
Summary updated