A robust approach to warped Gaussian process-constrained optimization

Johannes Wiebe, I. Cecílio, J. Dunlop, Ruth Misener

Abstract

This paper introduces a robust optimization framework for handling "black-box" constraints modeled by Warped Gaussian Processes (WGPs). By reformulating uncertain, aggregated constraints into deterministic approximations, the framework provides a way to guarantee safety and feasibility in complex systems—such as oil well drilling and production planning—even when the underlying data is non-Gaussian or limited.

Introduction

In many engineering and economic problems, constraints are often "black boxes" where exact formulas are unavailable, and only data from expensive simulations or physical experiments exist. Furthermore, these constraints are frequently aggregated; for example, the total wear on a drill bit is the sum of degradation across multiple drilling segments.

Traditional Bayesian optimization often struggles with these aggregated, uncertain constraints. Standard Gaussian Processes (GPs) assume that uncertainty follows a symmetric Gaussian distribution, but real-world data is often "warped" or skewed. If one assumes a simple Gaussian distribution when the reality is skewed, the resulting "safe" solution might actually lead to catastrophic failure. This paper seeks a way to "hedge" against this uncertainty using Robust Optimization (RO), ensuring constraints are met with a specific probability (e.g., 99% confidence).

Methodology

The authors combine three sophisticated mathematical tools to address these challenges:

1. Warped Gaussian Processes (WGPs)

Unlike standard GPs, WGPs use a monotonic warping function (often a sum of tanh functions) to map observations into a latent space where they exhibit Gaussian characteristics. This allows the model to capture non-standard noise and complex functional shapes that traditional models miss.

2. Robust Reformulation

The core innovation involves constructing an $\alpha$-confidence ellipsoid in the latent space of the WGP. By using Wolfe Duality, the authors transform the "semi-infinite" robust constraint—which must hold for all possible realizations in a set—into a single, deterministic constraint that standard solvers can handle.

Confidence ellipsoids in the latent space represent the uncertainty sets used for robust optimization.

3. Global Optimization Strategies

Because the warping function can introduce non-convexity, the authors developed a "divide and conquer" spatial branching algorithm to find global solutions. They also proposed an iterative a posteriori procedure to adjust the "conservatism" of the solution, ensuring it isn't excessively expensive while still meeting target feasibility.

Case Studies and Key Results

The paper evaluates the method using two primary case studies:

  • Production Planning: Using a synthetic dataset with non-uniform (heteroscedastic) noise, the WGP-based robust approach maintained high feasibility even as noise levels increased. In contrast, standard GP reformulations saw failure rates spike because they could not account for the non-Gaussian nature of the data.
  • Drill Scheduling: In an industrially relevant case study on mud motor degradation, the authors' "boundary heuristic" found optimal solutions significantly faster than total enumeration, making it viable for real-time scheduling.

Key insights show that "hedging" against uncertainty doesn't always lead to massive cost increases. In the drilling case, a 99%-confidence solution was often only slightly more expensive than a deterministic one, but provided a much higher safety margin against motor failure.

Limitations and Future Work

While effective, the method is inherently conservative, leading to a "safety tax" in the objective function. Additionally, solving the robust counterpart is more computationally intensive than nominal problems due to the bilinearities introduced during duality reformulation. Future research will explore more generalized frameworks for mixed-integer nonlinear programming (MINLP) with WGP constraints.

Summary updated

Abstract

Optimization problems with uncertain black-box constraints, modeled by warped Gaussian processes, have recently been considered in the Bayesian optimization setting. This work considers optimization problems with aggregated black-box constraints. Each aggregated black-box constraint sums several draws from the same black-box function with different decision variables as arguments in each individual black-box term. Such constraints are important in applications where, e.g., safety-critical measures are aggregated over multiple time periods. Our approach, which uses robust optimization, reformulates these uncertain constraints into deterministic constraints guaranteed to be satisfied with a specified probability, i.e., deterministic approximations to a chance constraint. While robust optimization typically considers parametric uncertainty, our approach considers uncertain functions modeled by warped Gaussian processes. We analyze convexity conditions and propose a custom global optimization strategy for non-convex cases. A case study derived from production planning and an industrially relevant example from oil well drilling show that the approach effectively mitigates uncertainty in the learned curves. For the drill scheduling example, we develop a custom strategy for globally optimizing integer decisions.
BibTeX
@article{wiebe2022robust,
  author = {Johannes Wiebe and I. Cecílio and J. Dunlop and Ruth Misener},
  title = {{A robust approach to warped Gaussian process-constrained optimization}},
  journal = {Math. Program.},
  year = {2022},
  doi = {10.1007/s10107-021-01762-8},
}
Figure