When designing a robotic exoskeleton or a visual prosthetic, how do you determine the "best" settings? Traditionally, an engineer might ask a user to rate a specific walking gait on a scale of 1 to 10. However, humans are notoriously inconsistent at providing absolute numerical scores. A "7" today might be a "5" tomorrow depending on fatigue, mood, or context.
This fundamental mismatch between human psychology and mathematical optimization has led to the development of Preferential Bayesian Optimization (PBO). Instead of asking for a score, PBO asks for a choice: "Do you prefer gait A or gait B?"
The Intuition of Relative Comparisons
The core insight, as explored by Kevin Jamieson and Ben Recht, is that while derivative-free optimization with noisy evaluations is theoretically slower than gradient-based methods, using "better-or-worse" comparisons can achieve the same optimal convergence rates as full numerical evaluations.
Humans are naturally better at relative judgment than absolute estimation. As Eric Brochu and Nando de Freitas argue in their foundational work, pairwise comparisons provide a much cleaner signal for modeling subjective preferences. By treating the human as a "comparator" rather than a "sensor," we can bypass the noise inherent in subjective ratings.
Mapping Preferences to a Latent Utility Function
To optimize based on "A > B" feedback, the system must infer an underlying objective function that it cannot directly see. This is typically done by modeling a latent utility function using a Gaussian Process (GP).
When a user chooses option A over option B, the system doesn't just record a binary data point; it updates the GP to increase the probability that the utility of A is higher than the utility of B. This allows the algorithm to build a probabilistic map of what the user likes across the entire search space. Tristan Fauvel and Matthew Chalk demonstrated this effectively in visual prosthetics, where they used these latent maps to translate visual scenes into electrical pulses that "looked" best to the patient.
Navigating High-Dimensional Spaces
The challenge of PBO is that pairwise comparisons are "low-information" data points. It takes many comparisons to map out a high-dimensional space. Researchers have developed several strategies to make this process efficient enough for real-time use:
- Dimensionality Reduction: Maegan Tucker and Aaron Ames developed the LINECOSPAR framework, which breaks high-dimensional gait optimization into a series of simple one-dimensional searches. This allowed an exoskeleton to find a personalized gait in just 30 trials.
- Latent Space Optimization: For complex tasks like apartment hunting, Michael Eichelbeck and colleagues perform PBO within the latent space of an autoencoder, using LLMs to set initial priors and solve the "cold start" problem.
- Advanced Acquisition Functions: Choosing which two items to compare next is critical. Raul Astudillo and Peter I. Frazier introduced qEUBO, a decision-theoretic approach that allows the system to compare more than two items at once (a "tournament"), significantly speeding up convergence.
From Gut Feeling to Mathematical Constraints
Perhaps the most powerful application of this research is in "capturing" expertise that cannot be written down. In robotics, experts often have a "gut feeling" about how a machine should move but struggle to write the exact cost function.
Sander De Witte and Guillaume Crevecoeur showed that PBO can "distill" this expert intuition into a data-driven cost function through simple duels. Similarly, Samuel M. Kolb demonstrated that optimization criteria and constraints can be learned directly from these preferences, turning subjective human values into rigorous mathematical objectives.
By prioritizing the way humans actually think—through comparison rather than calculation—these methods allow us to optimize systems that were previously too subjective or complex to tune.