Bayesian Optimisation (BO) is a powerful tool for finding the optimal settings in expensive, black-box experiments—such as tuning a chemical reaction or designing a new alloy. While AI excels at the underlying mathematics, human experts often possess deep domain knowledge or a "gut feeling" that the AI lacks.
The challenge is that this human knowledge is fluid. It is difficult to encode as a static mathematical formula at the start of an experiment, and it evolves as the expert observes new data. Existing methods for human-in-the-loop BO usually require experts to provide rigid specifications, which are difficult to articulate. This paper introduces Human-AI Teaming (HAT), a framework that allows human experts to guide BO through intuitive, occasional feedback. By using human corrections to refine the AI’s internal model rather than just adding data points, HAT achieves significantly faster convergence and remains robust even when human advice is imprecise.
How HAT Works
The authors propose two main strategies for incorporating human feedback into the BO loop:
- Rectifying Recommendations (HAT-RR): When the AI suggests a point to sample next, the expert can intervene and suggest an alternative point. The AI then updates its model's hyperparameters to ensure that the expert's suggested point has a higher "utility" (acquisition value) than its own original suggestion.
- Difference Maximisation (HAT-DM): The expert identifies a "good" region and a "bad" region. The AI then tunes its hyperparameters to maximize the difference in utility between these two regions while still fitting the observed data well.
Crucially, these methods do not just treat human input as additional data. Instead, they use the feedback as constraints on model selection. This forces the AI to choose a model of the world that aligns with the expert's intuition.
Key Insights: Hyperparameters as the Lever
The core theoretical breakthrough is that by incorporating expert feedback as constraints, the AI effectively ignores "unlikely" versions of the world. This narrows the space of possible functions the AI considers, which mathematically leads to tighter bounds on error (regret).
Instead of just adding the expert's point to the dataset, HAT changes the hyperparameters (like the lengthscale of a kernel). As seen below, this results in a much more focused model that aligns closely with the ground truth.
Furthermore, the authors build on psychological research suggesting that humans naturally behave like BO algorithms when searching. This allows the AI to "trust" human corrections as meaningful directional shifts rather than just random guesses.
Experimental Results
The researchers evaluated HAT using a mix of synthetic challenges (e.g., Ackley, Branin, Hartmann) and real-world engineering tasks, such as tuning Support Vector Machine (SVM) classifiers across 10 diverse datasets from the UCI repository.
- Superior Efficiency: Across almost all synthetic benchmarks, the HAT variants reached the global optimum faster than standard BO. The "simple regret" (the gap between the best-found value and the true optimum) dropped much more sharply for HAT methods.
- Real-World Gains: In the SVM tuning tasks, HAT-RR achieved the lowest classification error in 7 out of 10 datasets, proving its utility in practical machine learning workflows.
- Robustness to Error: Even when the human input was intentionally corrupted with noise to simulate mistakes (tested on Levy, Shubert, and Egg functions), the system still performed on par with or better than standard AI-only approaches.
Limitations and Future Work
While promising, the paper notes a few areas for further exploration. The study assumes the expert intervenes every few iterations; however, in real-world scenarios, human attention is a scarce resource, making the optimal "intervention schedule" an open question. Additionally, the current framework assumes the expert's internal model is somewhat similar to the AI's. Future work could investigate how to handle experts with radically different reasoning styles and focus on building intuitive user interfaces that make providing relative feedback seamless for non-AI experts.
Summary updated