Human-AI Collaborative Bayesian Optimisation

Arun Kumar Anjanapura Venkatesh, Santu Rana, Alistair Shilton, Svetha VenkateshNeurIPS 2022 · OpenReview

Bayesian Optimisation (BO) is a powerful tool for finding the optimal settings in expensive, black-box experiments—such as tuning a chemical reaction or designing a new alloy. While AI excels at the underlying mathematics, human experts often possess deep domain knowledge or a "gut feeling" that the AI lacks.

The challenge is that this human knowledge is fluid. It is difficult to encode as a static mathematical formula at the start of an experiment, and it evolves as the expert observes new data. Existing methods for human-in-the-loop BO usually require experts to provide rigid specifications, which are difficult to articulate. This paper introduces Human-AI Teaming (HAT), a framework that allows human experts to guide BO through intuitive, occasional feedback. By using human corrections to refine the AI’s internal model rather than just adding data points, HAT achieves significantly faster convergence and remains robust even when human advice is imprecise.

How HAT Works

The authors propose two main strategies for incorporating human feedback into the BO loop:

  1. Rectifying Recommendations (HAT-RR): When the AI suggests a point to sample next, the expert can intervene and suggest an alternative point. The AI then updates its model's hyperparameters to ensure that the expert's suggested point has a higher "utility" (acquisition value) than its own original suggestion.
  2. Difference Maximisation (HAT-DM): The expert identifies a "good" region and a "bad" region. The AI then tunes its hyperparameters to maximize the difference in utility between these two regions while still fitting the observed data well.

Crucially, these methods do not just treat human input as additional data. Instead, they use the feedback as constraints on model selection. This forces the AI to choose a model of the world that aligns with the expert's intuition.

Key Insights: Hyperparameters as the Lever

The core theoretical breakthrough is that by incorporating expert feedback as constraints, the AI effectively ignores "unlikely" versions of the world. This narrows the space of possible functions the AI considers, which mathematically leads to tighter bounds on error (regret).

Instead of just adding the expert's point to the dataset, HAT changes the hyperparameters (like the lengthscale of a kernel). As seen below, this results in a much more focused model that aligns closely with the ground truth.

Hyperparameter distribution obtained using HAT vs standard BO. HAT (blue) is much more concentrated around the ground truth (red) than standard BO (green).

Furthermore, the authors build on psychological research suggesting that humans naturally behave like BO algorithms when searching. This allows the AI to "trust" human corrections as meaningful directional shifts rather than just random guesses.

Experimental Results

The researchers evaluated HAT using a mix of synthetic challenges (e.g., Ackley, Branin, Hartmann) and real-world engineering tasks, such as tuning Support Vector Machine (SVM) classifiers across 10 diverse datasets from the UCI repository.

  • Superior Efficiency: Across almost all synthetic benchmarks, the HAT variants reached the global optimum faster than standard BO. The "simple regret" (the gap between the best-found value and the true optimum) dropped much more sharply for HAT methods.
  • Real-World Gains: In the SVM tuning tasks, HAT-RR achieved the lowest classification error in 7 out of 10 datasets, proving its utility in practical machine learning workflows.
  • Robustness to Error: Even when the human input was intentionally corrupted with noise to simulate mistakes (tested on Levy, Shubert, and Egg functions), the system still performed on par with or better than standard AI-only approaches.

Limitations and Future Work

While promising, the paper notes a few areas for further exploration. The study assumes the expert intervenes every few iterations; however, in real-world scenarios, human attention is a scarce resource, making the optimal "intervention schedule" an open question. Additionally, the current framework assumes the expert's internal model is somewhat similar to the AI's. Future work could investigate how to handle experts with radically different reasoning styles and focus on building intuitive user interfaces that make providing relative feedback seamless for non-AI experts.

Summary updated

Abstract

Abstract Human-AI collaboration looks at harnessing the complementary strengths of both humans and AI. We propose a new method for human-AI collaboration in Bayesian optimisation where the optimum is mainly pursued by the Bayesian optimisation algorithm following complex computation, whilst getting occasional help from the accompanying expert having a deeper knowledge of the underlying physical phenomenon. We expect experts to have some understanding of the correlation structures of the experimental system, but not the location of the optimum. The expert provides feedback by either changing the current recommendation or providing her belief on the good and bad regions of the search space based on the current observations. Our proposed method takes such feedback to build a model that aligns with the expert’s model and then uses it for optimisation. We provide theoretical underpinning on why such an approach may be more efficient than the one without expert’s feedback. The empirical results show the robustness and superiority of our method with promising efficiency gains.
BibTeX
@inproceedings{venkatesh2022human,
  author = {Arun Kumar Anjanapura Venkatesh and Santu Rana and Alistair Shilton and Svetha Venkatesh},
  title = {{Human-AI Collaborative Bayesian Optimisation}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=atd4X6U1jT},
}
Figure