In-Context Black-Box Optimization with Unreliable Feedback

Nicolas Samuel Blumer, Julien Martinelli, Samuel KaskiarXiv 2026 · arXiv

TL;DR

FICBO (Feedback-informed In-Context Black-Box Optimization) is a new Transformer-based framework that learns to optimize expensive experiments by leveraging "expert" advice—even when that advice is biased or misleading. By pretraining on a diverse prior of unreliable feedback sources, the model learns to calibrate its trust in real-time, outperforming traditional Bayesian Optimization methods that struggle with inconsistent side information.

What Problem Does This Paper Address?

In fields like drug discovery or aerospace engineering, running experiments is incredibly expensive. To speed things up, researchers often use "side information" like cheap simulators, pretrained models, or human intuition. However, this feedback is a double-edged sword: a simulator might be accurate only in certain regions, or an expert might have a systematic bias.

Traditional Bayesian Optimization (BO) methods usually treat this feedback as a static prior, requiring users to manually tune how much they "trust" the source. If you trust a bad source too much, your optimization fails; if you trust it too little, you waste time. Furthermore, standard in-context optimizers (which learn to optimize by looking at history) typically ignore this auxiliary feedback entirely. The authors tackle the challenge of trust calibration: how can an optimizer automatically infer the reliability of a feedback source on the fly and use it effectively?

Methods

The authors introduce FICBO, which treats optimization as an "in-context" learning problem. Instead of fitting a new model for every task, they pretrain a Transformer on millions of synthetic optimization episodes.

The key innovation is the Feedback Prior. During training, the model is exposed to "experts" with varying levels of quality:

  • Globally useful: High correlation with the true objective.
  • Locally useful: Accurate only in specific parts of the design space.
  • Misleading: Systematically biased or completely wrong.
Figure 1: Feedback-aware amortized BO (FICBO) and related work. During training, tasks are sampled from a feedback prior with auxiliary sources of varying reliability. A transformer model learns both posterior prediction and query selection from standard features and feedback. At deployment, it conditions on the optimization history and the available feedback to guide search.

To make the model robust, the authors applied five types of signal degradation to the training feedback: constant shifts, smooth GP-based biases, local distortions, random noise, and "catastrophic failure" (replacing feedback with random values).

Figure 2: Illustration of five signal degradation bias types applied to a feedback GP.

At test time, FICBO receives a set of candidates and their corresponding feedback. As it collects real data points, it compares the observed values to the feedback. The Transformer architecture allows it to implicitly "reason" about the source's reliability and adjust its search strategy accordingly.

Datasets

The model was evaluated on a mix of synthetic and real-world benchmarks:

  • Synthetic Benchmarks: 13 standard functions (e.g., Ackley, Rosenbrock, Hartmann 3D) with simulated "marginal experts" and XGBoost-based biased experts.
  • Multi-fidelity Branin: A classic test case for multi-fidelity optimization with adjustable discrepancy levels.
  • Aerodynamics (Airfoil): Optimizing the lift-to-drag ratio of a wing using a cheap, lower-accuracy neural surrogate as feedback.
  • ML Hyperparameter Tuning: Optimizing neural architectures (NAS-Bench-201) where CIFAR-10 accuracy serves as a cheap proxy for ImageNet performance.
  • Chemical Reactor Optimization: A task where reactant conversion is a cheap but structurally biased proxy for product yield.

Key Insights and Ideas

  • Trust is Contextual: The model doesn't just decide if an expert is "good" or "bad" overall. It learns to identify where the expert is useful. In the chemical reactor task, it correctly identifies that the "conversion" proxy is useful early on but becomes misleading near the optimum.
  • Robustness through Exposure: By seeing "catastrophic" expert failures during pretraining, FICBO develops a "skeptical" stance. When feedback contradicts observed data, the model quickly discounts the feedback and reverts to standard search, preventing it from being led astray.
  • Amortization Saves Time: Because the "reasoning" is baked into the Transformer's weights, FICBO can make high-quality decisions in a single forward pass, avoiding the expensive internal loops required by traditional BO.

Key Results

FICBO consistently achieved lower cumulative regret (a measure of how much "value" was lost during search) compared to baselines like Vanilla BO and $\pi$BO.

Figure 3: Performance across synthetic benchmarks and multi-fidelity Branin tasks. Top row: normalized best objective found so far. Bottom row: normalized cumulative regret. (a–b) Results averaged over 13 benchmark functions (Sphere, Ackley, Rastrigin, Rosenbrock, Schwefel, Levy, 2D and 3D versions, and Hartmann 3D) under two feedback regimes: marginal expert (observes only the last input dimension) and XGBoost expert (data-driven but informationally limited). (c–e) Results for multi-fidelity Branin function under three discrepancy levels (a ∈{0, 0.5, 0.75}), where larger a increases the mismatch between the feedback and true objective. Mean ± 95% bootstrap confidence intervals. For all functions, the optimizer model has access to the first two feature dimensions.

In the Airfoil Lift experiment—a particularly tricky case where the feedback source ignores drag—standard feedback-aware methods failed by over-optimizing for lift. FICBO, however, detected the mismatch after just a few evaluations and successfully found the true optimal lift-to-drag ratio.

Limitations and Future Work

  • Prior Coverage: The model's performance depends heavily on the diversity of the pretraining prior. If a real-world expert fails in a way the model never saw during training, it might still be fooled.
  • Fixed Feedback: Currently, the feedback source is fixed at the start. In many real-world scenarios, we might want to "query" the expert or simulator actively, which would require a more complex acquisition mechanism.
  • High-Dimensional Scaling: The experiments were limited to relatively low-dimensional design spaces (up to 6D). Scaling Transformer-based BBO to very high dimensions remains an open research question.

Future work could involve training on even richer objective priors, such as those derived from large-scale tabular datasets, to create a truly "foundation model" for experimental design.

Summary updated

Abstract

Black-box optimization in science and engineering often comes with side information: experts, simulators, pretrained predictors, or heuristics can suggest which candidates look promising. This information can accelerate search, but it can also be biased, input-dependent, or misleading. Feedback-aware BO methods typically handle one task at a time, limiting their ability to generalize over multiple sources of feedback. In-context optimizers address cross-task adaptation, but usually assume that optimization history is the only available signal at test time. We study feedback-informed in-context black-box optimization (FICBO), where a pretrained optimizer conditions on both the observed history and cheap auxiliary feedback for the current candidate set. We introduce a structured feedback prior that models how feedback sources vary in their access, relevance, and distortion relative to the true objective, and use it to pretrain a feedback-aware transformer. At test time, the model estimates source reliability in context by comparing observed objective values with auxiliary signals, improving query selection. On synthetic and real-world tasks, FICBO effectively exploits informative feedback while remaining robust to weak or misleading sources, improving over other baselines. Empirical investigations further illustrate how the model perceives test-time sources, offering insights into its interpretability and decision-making process.
BibTeX
@misc{blumer2026context,
  author = {Nicolas Samuel Blumer and Julien Martinelli and Samuel Kaski},
  title = {{In-Context Black-Box Optimization with Unreliable Feedback}},
  year = {2026},
  eprint = {2605.06187},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.06187},
}
Figure