In engineering, design, and machine learning, the most efficient solutions rarely sit comfortably in the middle of a safe zone. Whether optimizing the structural integrity of a vessel (Nobukawa et al.) or tuning the parameters of an electric motor (Loka et al.), the global optimum is almost always pushed to the absolute limit of safety, budget, or physical constraints. These are boundary-proximal optima.
Traditional optimization algorithms struggle in these environments. Standard constrained Bayesian optimization, for instance, typically computes an acquisition function and multiplies it by the probability of feasibility. Near a complex boundary, this probability drops precipitously, making the algorithm overly timid and causing it to stall far from the true edge. To uncover the highest-performing solutions, a growing body of research focuses on active constraint navigation—treating the boundaries not as terminal walls to avoid, but as primary geometric features to map and traverse.
The Informational Value of Infeasibility
Why would an optimizer ever choose to sample a point that violates a constraint? Intuitively, if you are trying to map a hidden wall in a dark room, bumping into the wall tells you exactly where it stands.
Picheny formalized this using a Stepwise Uncertainty Reduction (SUR) approach. Instead of merely looking for valid points with high rewards, the algorithm selects points specifically to reduce the overall statistical uncertainty regarding the location of the constraint boundary.
This concept becomes powerful when paired with multi-step planning. As Lam et al. and Zhang et al. demonstrate, myopic (one-step-ahead) optimizers inherently avoid infeasible regions because they offer zero immediate reward. However, a lookahead mechanism recognizes that probing an invalid region provides critical topological information. By sacrificing an immediate valid evaluation to pinpoint the boundary's exact contour, the optimizer can confidently execute highly aggressive, optimal moves along that boundary in subsequent steps.
Myopic Search: [Optimum] ✖ | (Stalls safely inside)
Lookahead Search: [Current] -> | -> ✖ (Probes invalid zone) -> [Optimum] (Lands on edge)
Sliding Along the Constraint Wall
Once an algorithm reaches a boundary, it must be able to move along it without falling into forbidden territory or bouncing back into the interior.
Standard local search methods often treat boundaries as terminal points, getting stuck as soon as a constraint is triggered. To overcome this, Sakuma et al. introduced a method that dynamically incorporates active constraints as linear inequalities directly into the local direction-finding step. Instead of stopping, the algorithm calculates a "Pareto descent direction" that slides precisely along the intersection of the constraint surfaces.
In scenarios where out-of-bounds evaluations are strictly prohibited—such as real-world physical systems operating under strict safety limits—algorithms cannot afford to cross the line even for exploration. To navigate this, Achddou et al. utilize approximate tangent cones. By selecting search directions from a positive spanning set that respects the local geometry of the boundary, the algorithm guarantees safety while aggressively probing the absolute edge of the feasible domain.
Boundary-Aware Surrogates and Active Learning
To navigate boundaries effectively, the underlying statistical models must natively understand that the search space has edges. Standard Gaussian Process kernels treat all regions of a search space uniformly, which blurs the sharp local characteristics of boundary-proximal optima.
To resolve this, recent work on Bayesian Optimization over Bounded Domains with the Beta Product Kernel introduces non-stationary covariance functions. By leveraging the properties of Beta distributions, these kernels naturally adapt to the boundaries of a bounded domain, focusing modeling resources where the global optimum is most likely to hide.
When constraints are entirely unknown and black-box, frameworks like COBALT, proposed by Chen et al., elevate constraint mapping to a first-class objective. Rather than treating constraints as passive filters, they actively learn the intersection of high-reward regions and feasibility boundaries. By focusing exploration strictly on the feasibility boundaries of high-performing candidates—a strategy also mirrored in the bandit setting by Lindner et al.—these methods avoid the wastefulness of mapping the entire constraint space, zooming in precisely where the optimal trade-offs reside.
Key Foundational Work
An Adaptive Algorithm for Constrained Optimization Problems (2000) — contributed 2 concept elements to this direction.
An Efficient Method by Introducing Concept of Generalized Design (1998) — contributed 2 concept elements to this direction.