TL;DR
FICBO (Feedback-informed In-Context Black-Box Optimization) is a new Transformer-based framework that learns to optimize expensive experiments by leveraging "expert" advice—even when that advice is biased or misleading. By pretraining on a diverse prior of unreliable feedback sources, the model learns to calibrate its trust in real-time, outperforming traditional Bayesian Optimization methods that struggle with inconsistent side information.
What Problem Does This Paper Address?
In fields like drug discovery or aerospace engineering, running experiments is incredibly expensive. To speed things up, researchers often use "side information" like cheap simulators, pretrained models, or human intuition. However, this feedback is a double-edged sword: a simulator might be accurate only in certain regions, or an expert might have a systematic bias.
Traditional Bayesian Optimization (BO) methods usually treat this feedback as a static prior, requiring users to manually tune how much they "trust" the source. If you trust a bad source too much, your optimization fails; if you trust it too little, you waste time. Furthermore, standard in-context optimizers (which learn to optimize by looking at history) typically ignore this auxiliary feedback entirely. The authors tackle the challenge of trust calibration: how can an optimizer automatically infer the reliability of a feedback source on the fly and use it effectively?
Methods
The authors introduce FICBO, which treats optimization as an "in-context" learning problem. Instead of fitting a new model for every task, they pretrain a Transformer on millions of synthetic optimization episodes.
The key innovation is the Feedback Prior. During training, the model is exposed to "experts" with varying levels of quality:
- Globally useful: High correlation with the true objective.
- Locally useful: Accurate only in specific parts of the design space.
- Misleading: Systematically biased or completely wrong.
To make the model robust, the authors applied five types of signal degradation to the training feedback: constant shifts, smooth GP-based biases, local distortions, random noise, and "catastrophic failure" (replacing feedback with random values).
At test time, FICBO receives a set of candidates and their corresponding feedback. As it collects real data points, it compares the observed values to the feedback. The Transformer architecture allows it to implicitly "reason" about the source's reliability and adjust its search strategy accordingly.
Datasets
The model was evaluated on a mix of synthetic and real-world benchmarks:
- Synthetic Benchmarks: 13 standard functions (e.g., Ackley, Rosenbrock, Hartmann 3D) with simulated "marginal experts" and XGBoost-based biased experts.
- Multi-fidelity Branin: A classic test case for multi-fidelity optimization with adjustable discrepancy levels.
- Aerodynamics (Airfoil): Optimizing the lift-to-drag ratio of a wing using a cheap, lower-accuracy neural surrogate as feedback.
- ML Hyperparameter Tuning: Optimizing neural architectures (NAS-Bench-201) where CIFAR-10 accuracy serves as a cheap proxy for ImageNet performance.
- Chemical Reactor Optimization: A task where reactant conversion is a cheap but structurally biased proxy for product yield.
Key Insights and Ideas
- Trust is Contextual: The model doesn't just decide if an expert is "good" or "bad" overall. It learns to identify where the expert is useful. In the chemical reactor task, it correctly identifies that the "conversion" proxy is useful early on but becomes misleading near the optimum.
- Robustness through Exposure: By seeing "catastrophic" expert failures during pretraining, FICBO develops a "skeptical" stance. When feedback contradicts observed data, the model quickly discounts the feedback and reverts to standard search, preventing it from being led astray.
- Amortization Saves Time: Because the "reasoning" is baked into the Transformer's weights, FICBO can make high-quality decisions in a single forward pass, avoiding the expensive internal loops required by traditional BO.
Key Results
FICBO consistently achieved lower cumulative regret (a measure of how much "value" was lost during search) compared to baselines like Vanilla BO and $\pi$BO.
In the Airfoil Lift experiment—a particularly tricky case where the feedback source ignores drag—standard feedback-aware methods failed by over-optimizing for lift. FICBO, however, detected the mismatch after just a few evaluations and successfully found the true optimal lift-to-drag ratio.
Limitations and Future Work
- Prior Coverage: The model's performance depends heavily on the diversity of the pretraining prior. If a real-world expert fails in a way the model never saw during training, it might still be fooled.
- Fixed Feedback: Currently, the feedback source is fixed at the start. In many real-world scenarios, we might want to "query" the expert or simulator actively, which would require a more complex acquisition mechanism.
- High-Dimensional Scaling: The experiments were limited to relatively low-dimensional design spaces (up to 6D). Scaling Transformer-based BBO to very high dimensions remains an open research question.
Future work could involve training on even richer objective priors, such as those derived from large-scale tabular datasets, to create a truly "foundation model" for experimental design.
Summary updated