Bayesian Quantile and Expectile Optimisation

Léonard Torossian, Victor Picheny, Nicolas DurrandearXiv 2020 · OpenReview

TL;DR

This paper introduces a novel Bayesian optimization (BO) framework for optimizing risk-averse criteria like quantiles and expectiles, particularly in scenarios with noisy and unevenly distributed data (heteroscedastic noise). They achieve this by proposing new probabilistic models and efficient acquisition strategies that outperform existing methods, especially in complex, high-dimensional, and large-batch settings.

What Problem Does This Paper Address?

Traditional Bayesian optimization excels at finding the maximum of a function when we can reliably observe its value. However, many real-world problems require optimizing not just the average outcome, but also considering the risks associated with extreme events. For instance, in finance, one might want to minimize the 95% quantile of potential losses, or in engineering, ensure a system operates reliably even under worst-case scenarios. Existing BO methods often struggle with these "risk-averse" objectives because they typically focus on optimizing the expected value and may not accurately model the tails of a probability distribution. Furthermore, many real-world functions are "stochastic black-box functions," meaning their output is noisy, and the noise level can vary depending on the input (heteroscedasticity). Directly optimizing quantiles or expectiles in such settings is challenging because it requires accurately estimating the distribution of outcomes, not just their average, and doing so efficiently with limited, noisy observations.

Methods

The core of this work lies in two main innovations:

  1. Novel Probabilistic Models for Quantile/Expectile Regression: The authors propose a new way to model quantiles and expectiles using two latent Gaussian Processes (GPs). One GP models the conditional quantile/expectile itself, while a second GP models the scale of the noise. This is crucial for handling heteroscedastic noise, where the noise level changes with the input. They use asymmetric likelihood functions (asymmetric Laplace for quantiles, asymmetric Gaussian for expectiles) that are naturally suited for these risk measures. A key advantage is that their model doesn't assume a specific parametric form for the noise distribution, making it more flexible. They employ sparse variational inference (specifically, a Sparse Variational Gaussian Process - SVGP) to handle potentially large datasets efficiently.
  2. New Bayesian Optimization Acquisition Strategies: To effectively explore the input space and find the optimal quantile/expectile, they introduce two new acquisition functions tailored to their models:
  3. Thompson Sampling (TS): This strategy leverages the posterior distribution of their GP models to draw samples and select the input that maximizes the sampled quantile/expectile. It's inherently parallelizable, making it suitable for large batches.
  4. Quantile GIBBON (Q-GIBBON): This is an adaptation of the Max-Value Entropy Search (MES) approach. It aims to select batches of points that maximally reduce uncertainty about the maximum quantile/expectile value. It's designed to handle large batches and uses a moment-matching approximation for tractability.

Datasets

The paper evaluates their methods on:

  • Synthetic Problems: They generate synthetic datasets using the Generalised Lambda Distribution (GLD). This distribution is highly flexible and allows them to create black-box functions with varying dimensionality, heteroscedasticity, and non-Gaussianity, mimicking complex real-world scenarios.
  • Lunar Lander: A popular reinforcement learning benchmark known for its noisy and non-Gaussian reward structure.
  • FEL Tuning: A real-world optimization task inspired by Free-Electron Laser (FEL) calibration, involving a high-dimensional problem with significant observation noise and a need for risk-averse optimization.

Key Insights and Ideas

  • Heteroscedasticity is Key for Risk Measures: The authors demonstrate that accurately modeling the varying noise level (heteroscedasticity) is critical for effectively optimizing quantiles and expectiles, especially in complex scenarios. Simple models that assume constant noise fail to capture the true uncertainty.
  • Direct Optimization is Superior: Their approach directly optimizes for the quantile or expectile, avoiding the need to replicate observations or assume a parametric noise model, which simplifies the process and improves efficiency.
  • Large Batches are Essential: For quantile/expectile optimization, where more data is generally needed to accurately estimate tail behavior, large batch sizes are crucial. Their proposed methods are specifically designed to handle this.
  • Trade-offs in Acquisition Functions: While Q-GIBBON is theoretically motivated by information theory, Thompson Sampling proves to be a strong practical performer, especially in higher dimensions, due to its simplicity and parallelizability.

Key Results

  • Outperforms State-of-the-Art: Across various synthetic and real-world problems, their proposed Bayesian optimization strategies (using their novel GP models) consistently outperform existing baselines like standard GP regression with EI (GPR-EI) and heteroscedastic GP (HetGP).
  • Significant Gains in Complex Settings: The performance gap widens considerably with increasing problem dimension, batch size, and non-Gaussianity of the noise.
  • Well-Calibrated Confidence Intervals: Figure 1 visually demonstrates that their model produces more accurate and better-calibrated confidence intervals for quantiles compared to previous methods, especially in regions with high heteroscedasticity.
  • Efficiency for Large Batches: Their methods are shown to be effective in large-batch settings (e.g., B=50), which are often impractical for older BO algorithms.

Limitations and Future Work

  • High-Dimensional Q-GIBBON: The authors note that the performance of Q-GIBBON might degrade in very high-dimensional problems due to the reliance on sampling for its acquisition function. Extending information-theoretic BO to such scenarios remains an open question.
  • Computational Cost: While their methods are efficient, complex models and large datasets can still incur significant computational costs, especially for hyperparameter tuning.
  • Further Exploration of Acquisition Functions: The paper focuses on TS and Q-GIBBON. Future work could explore other acquisition strategies tailored to their quantile/expectile models, potentially combining the strengths of different approaches.
  • Expectile Performance: While the paper introduces models for both quantiles and expectiles, the experimental focus is primarily on quantiles. Further validation of their expectile optimization capabilities would be valuable.

Summary updated

Abstract

Bayesian optimisation (BO) is widely used to optimise stochastic black box functions. While most BO approaches focus on optimising conditional expectations, many applications require risk-averse strategies and alternative criteria accounting for the distribution tails need to be considered. In this paper, we propose new variational models for Bayesian quantile and expectile regression that are well-suited for heteroscedastic noise settings. Our models consist of two latent Gaussian processes accounting respectively for the conditional quantile (or expectile) and the scale parameter of an asymmetric likelihood functions. Furthermore, we propose two BO strategies based on max-value entropy search and Thompson sampling, that are tailored to such models and that can accommodate large batches of points. Contrary to existing BO approaches for risk-averse optimisation, our strategies can directly optimise for the quantile and expectile, without requiring replicating observations or assuming a parametric form for the noise. As illustrated in the experimental section, the proposed approach clearly outperforms the state of the art in the heteroscedastic, non-Gaussian case.
BibTeX
@misc{torossian2020bayesian,
  author = {Léonard Torossian and Victor Picheny and Nicolas Durrande},
  title = {{Bayesian Quantile and Expectile Optimisation}},
  year = {2020},
  eprint = {2001.04833},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2001.04833},
}
Figure