When optimizing complex, expensive, or non-differentiable systems—such as physical simulations, chemical processes, or quantized neural networks—we often rely on a surrogate model (or proxy) to guide our search. The standard workflow seems straightforward: gather data from the true system, train a machine learning model to minimize prediction error, and then use gradient descent on this surrogate to find the optimal design.
Yet, this approach frequently fails. Surrogates with exceptional prediction accuracy can still mislead optimizers into poor solutions, spurious local minima, or catastrophic divergence.
The reason lies in a fundamental geometric mismatch: optimization algorithms do not care about a model's values; they care about its gradients. To successfully guide an optimization loop, a surrogate must satisfy a descent condition—its gradient field must align with the gradient field of the true system.
The Pitfall of Value-Based Surrogates
Consider training a surrogate model $\hat{f}(x)$ to approximate an expensive black-box function $f(x)$. Standard regression algorithms minimize a value-based loss, such as Mean Squared Error (MSE), ensuring that $\hat{f}(x) \approx f(x)$ across the training distribution.
However, an optimization algorithm updates parameters by taking steps proportional to the surrogate's gradient, $\nabla \hat{f}(x)$. If the surrogate accurately predicts the function's value but fails to capture its derivatives, the resulting vector field becomes distorted. As Junlong Lyu et al. point out, minimizing value-based loss alone frequently yields inaccurate vector fields, causing gradient-based optimization to stall in local optima or suffer from gradient explosions.
This discrepancy is especially pronounced in high-dimensional scientific computing. For instance, when optimizing systems governed by partial differential equations, Luo et al. observe that neural operators can exhibit high standard generalization accuracy while remaining completely ineffective inside an optimization loop. Without explicit regularization of the network's derivatives, the inaccurate gradient approximations routinely guide the optimizer into spurious local minima.
The Geometry of the Descent Condition
To understand why an optimization loop succeeds or fails, we must look at the geometry of the update step. For an optimizer taking a step along the surrogate gradient direction $-\nabla \hat{f}(x)$ to successfully decrease the true objective $f(x)$, the step must move in a direction that forms an acute angle with the true negative gradient $-\nabla f(x)$.
Mathematically, this requires the inner product of the two gradient vectors to be strictly positive:
$$\langle \nabla f(x), \nabla \hat{f}(x) \rangle > 0$$
This requirement is known as the surrogate descent condition.
True Negative Gradient: ─────────────────────────> −∇f(x)
▲
/ (Acute Angle: Positive Inner Product)
/
Surrogate Step Direction: ─────────> −∇f̂(x)
When this condition is met, the optimizer is guaranteed to make progress toward a local minimum of the true function, even if the surrogate's value predictions are imperfect.
This principle explains the success of heuristic optimization shortcuts. For example, in training activation-quantized neural networks where true gradients are zero almost everywhere, the Straight-Through Estimator (STE) replaces the discontinuous step function with a smooth identity mapping in the backward pass. Jiancheng Lyu et al. demonstrate that the STE works precisely because the expected proxy direction correlates positively with the true population gradient. This alignment ensures that the optimization process consistently moves toward lower loss regions despite the mathematical inconsistency between the forward and backward passes.
To evaluate the reliability of an approximate gradient, Gould et al. formalize this geometric relationship by measuring the cosine similarity between the approximate gradient and the exact gradient derived from implicit differentiation. A positive cosine similarity serves as a mathematical guarantee that the approximation provides a valid descent direction.
In the domain of derivative-free evolutionary strategies, Youhei Akimoto proves that a surrogate-assisted update maintains a proper descent direction as long as the alignment between the surrogate's guidance and the ideal update exceeds the noise introduced by the approximation. This provides a theoretical threshold for surrogate accuracy, below which the optimizer risks divergence.
Bounding Optimization Risk via Gradient Matching
The descent condition is particularly difficult to maintain in offline black-box optimization, where an optimizer uses a static dataset to find an optimal design. As the optimizer updates the inputs to maximize performance, it naturally pushes the surrogate model into out-of-distribution (OOD) regions where the model has never seen data. Standard value-based regression models typically collapse in these regions, providing highly inaccurate gradients.
To solve this, recent work shifts the focus of surrogate training from value replication to explicit gradient matching. Minh Hoang et al. prove that the worst-case optimization performance gap—the difference between the true optimum and the optimum found via the surrogate—is directly bounded by the maximum norm difference between the surrogate's gradients and the true function's gradients.
By training surrogate models to minimize this gradient discrepancy directly, the optimization process becomes significantly more robust to distribution shifts. In a companion work, Minh Hoang et al. introduce an algorithmic framework (MATCH-OPT) that leverages the fundamental theorem of line integrals to estimate and match these latent gradient fields from static datasets. Because the model explicitly learns correct directional derivatives rather than point-wise scalar values, it retains its predictive utility and provides stable optimization signals even when pushed deep into unfamiliar OOD territory.
Distorting the Landscape: Lessons from Constraints and Physics
When surrogate gradients fail to align with the true system, the entire optimization landscape becomes warped. In parameter estimation and inverse problems, Behroozi et al. show that if a surrogate model fails to correctly represent the relationship between input shifts and output sensitivities, the search landscape becomes distorted. Optimization algorithms navigating this distorted terrain inevitably converge on incorrect physical constants. To prevent this, the authors introduce Sensitivity-Constrained Fourier Neural Operators, forcing the network to learn both the system state and its exact derivatives.
This insight echoes classic findings from chemical process engineering. Decades ago, Grossmann et al. established that using simplified models within optimization loops can lead to complete convergence failures or infinite cycling if the simplified model's gradients fail to match those of the rigorous model at the true optimum.
Whether optimizing fractional functions by simplifying complex denominators (Ganzhao Yuan), tuning pre-processing filters for conventional image codecs using differentiable proxy models (Klopp et al.), or identifying interesting data regions with evolutionary algorithms (Savva et al.), the underlying rule remains identical. For an embedded approximation to successfully guide an optimizer to the true global optimum, minimizing value error is a secondary concern; aligning the gradients is what guarantees descent.
Key Foundational Work
A note on approximation techniques used for process optimization (1985) — contributed 10 concept elements to this direction.
Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets (2019) — contributed 1 concept elements to this direction.