Transforming Stochastic Objectives into Deterministic Surfaces

This direction is no longer present in the current build. Its last version stays readable and citable here.

Version 1 of 115 papers19 conceptsfrozen 2026-10-04

Optimization in the real world is rarely a clean, algebraic exercise. Often, the function we want to maximize is hidden behind a "black box" simulation or a complex expectation over random variables. In these settings, traditional optimization faces a fundamental hurdle: noise. If every time you evaluate a function you get a slightly different answer, the "landscape" of the problem chatters and shifts, making it nearly impossible for an optimizer to find a true peak.

A powerful recurring strategy across control theory, Bayesian optimization, and variational inference is to bypass this noise by fixing the randomness. By transforming a stochastic problem into a deterministic proxy, researchers can apply high-performance solvers that would otherwise fail in noisy environments.

The Problem of Shifting Landscapes

Consider training a controller for a helicopter, as explored by Andrew Y. Ng and colleagues in Autonomous Helicopter Flight via Reinforcement Learning. The goal is to find a policy that maximizes a reward. However, the helicopter's environment is stochastic—wind gusts and sensor noise mean that the same control inputs lead to different outcomes every time.

If you use a standard stochastic optimizer, the reward surface is constantly vibrating. An optimizer might think it has found a path uphill, only for the "hill" to disappear in the next simulation run due to a different random seed. This variance forces practitioners to use slow, conservative methods like Stochastic Gradient Descent (SGD), which require many samples to average out the noise.

Fixing the Dice

The core insight of this research direction—often formalized as Sample Average Approximation (SAA)—is to "fix the dice." Instead of drawing new random samples for every function evaluation, we choose a large, fixed set of random seeds (or "base samples") at the start and use them for every subsequent evaluation during the optimization process.

As Maximilian Balandat and Benjamin Letham demonstrate in the BoTorch framework, this simple act of fixing base samples transforms a jagged, stochastic acquisition surface into a smooth, deterministic one.

When the randomness is fixed, the objective function becomes a consistent "proxy" for the true expectation. Because this proxy is deterministic, we can compute its exact gradients and even its curvature (Hessians). This allows the use of powerful quasi-Newton optimizers like L-BFGS, which converge much faster and more reliably than stochastic methods.

Applications Across Domains

This "derandomization" logic appears under various names across different fields:

  • Bayesian Optimization: In Differentiable Expected Hypervolume Improvement, Balandat and Samuel Daulton use SAA to make complex multi-objective acquisition functions differentiable. This enables parallel optimization of multiple points at once, a task that is computationally prohibitive with traditional methods.
  • Variational Inference: Standard Black Box Variational Inference (BBVI) is often plagued by high-variance gradients. Ryan Giordano and Tamara Broderick introduced Deterministic ADVI (DADVI) in Black Box Variational Inference with a Deterministic Objective. By using a fixed set of Monte Carlo samples, they create a stable objective that allows for second-order optimization and more accurate uncertainty estimates.
  • Decision Making: In the context of stochastic dynamic programs, Nan Chen and colleagues use Information Relaxation to convert complex decision paths into deterministic "inner problems." By fixing specific sample paths of randomness, they can use off-the-shelf deterministic solvers to find near-optimal policies in otherwise intractable environments.

Why It Works: Stability and Curvature

The intuition for why this works lies in the trade-off between bias and variance.

A single evaluation with a fixed seed is a biased estimate of the true average. However, by using a sufficiently large number of fixed samples, the bias becomes negligible. In exchange for this tiny bias, we gain massive stability. As Antoine Grosnit and Alexander I. Cowen-Rivers argue in Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?, fixing samples ensures the optimizer converges to a consistent local optimum rather than wandering aimlessly in the noise.

Furthermore, this approach exploits the geometry of the problem. In A Derandomization Framework for Structure Discovery, Nikos Tsikouras and Christos Tzamos show that seeking second-order stationary points in these deterministic landscapes allows algorithms to discover hidden structures, such as low-rank matrices in neural networks, by following the natural curvature of the objective.

By treating randomness as a fixed coordinate rather than a moving target, these researchers have turned "messy" stochastic problems into structured challenges that modern, high-speed deterministic solvers can master.

Papers in this version

History & citationVersion 1 · 2026-10-04

No longer present in the current build · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_transforming_stochastic_objectives_into__ad92ca54,
  title = {Transforming Stochastic Objectives into Deterministic Surfaces},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/transforming-stochastic-objectives-into-deterministic-surfaces/rdr_ad92ca54040042af8273cf4328276696},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_transforming_stochastic_objectives_into__ad92ca54_v1,
  title = {Transforming Stochastic Objectives into Deterministic Surfaces},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/transforming-stochastic-objectives-into-deterministic-surfaces/rdr_ad92ca54040042af8273cf4328276696/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}