Incremental Decision Making Under Risk with the Weighted Expected Utility Model

Hugo Gilbert, Nawal Benabbou, Patrice Perny, Olivier Spanjaard, Paolo ViappianiIJCAI 2017 · DOI

TL;DR

This paper introduces an efficient method for eliciting a decision-maker's preferences under risk using the Weighted Expected Utility (WEU) model. By employing a novel incremental approach and representing utility and weighting functions as monotone splines, the authors significantly reduce the number of queries needed to identify near-optimal choices, making complex decision support systems more practical.

What Problem Does This Paper Address?

Decision-making under risk, where individuals choose between uncertain outcomes (lotteries), is a fundamental problem. The standard Expected Utility (EU) model, while elegant, fails to capture certain observed behaviors like Allais' paradox. The Weighted Expected Utility (WEU) model offers a more descriptive alternative by introducing an additional weighting function. However, eliciting the parameters of WEU (two functions, $u$ and $w$) is challenging due to the complexity and the sheer number of potential questions required for a full elicitation. This paper tackles the problem of efficiently identifying a near-optimal lottery for a decision-maker within a given set of options, without needing to fully specify the WEU model. The goal is to find a practical elicitation strategy that balances descriptive accuracy with computational feasibility.

Methods

The core of this paper is an incremental elicitation procedure for the WEU model. Instead of asking a fixed sequence of questions, their method adaptively selects the next query to maximize information gain. Key methodological contributions include:

  • WEU Model Foundation: The paper builds upon the WEU model, which generalizes EU and can account for phenomena like Allais' paradox. Preferences are defined by $p \succ q \iff u(p)w(q) - u(q)w(p) \ge 0$.
  • Monotone Spline Representation: To manage the complexity of eliciting the utility function ($u$) and the weighting function ($w$), the authors represent them as monotone cubic I-splines. This allows for a compact representation of potentially complex functions using a limited number of parameters (coefficients of the splines) and ensures the required monotonicity properties ( $u$ non-decreasing, $w$ non-increasing).
  • Adaptive Query Generation: The procedure iteratively refines the uncertainty about $u$ and $w$. At each step, it computes a "maxmin" lottery ($p^*$) that represents the best choice in the worst-case scenario, and a "challenger" lottery ($p^*$) that might lead to the largest loss if $p^*$ is chosen. Queries are then generated to reduce the uncertainty around these critical lotteries.
  • Decoupled Elicitation: The method cleverly decouples the elicitation of $u$ and $w$ by using specific query types (uQuery and wQuery) that generate linear constraints on the spline coefficients for $u$ and $w$ independently. This simplifies the optimization problem.
  • Maxmin Optimization: The selection of the next query is guided by a maxmin optimization criterion, aiming to find the lottery that performs best under the worst-case assumptions about the DM's unknown preferences.

Datasets

Not discussed in the paper. The experiments simulate decision-maker responses using randomly generated WEU models based on spline functions.

Key Insights and Ideas

  • WEU's Descriptive Power: The paper reinforces that WEU is a powerful model capable of explaining observed deviations from EU, like Allais' paradox, while retaining desirable normative properties like transitivity and betweenness.
  • Splines as a Practical Representation: Using monotone splines for $u$ and $w$ is a crucial innovation. It provides a flexible yet constrained representation that significantly reduces the dimensionality of the elicitation problem and allows for efficient computation of function values and their bounds.
  • Incremental Elicitation is Key: The core insight is that a full elicitation is often unnecessary and computationally prohibitive. An incremental, adaptive approach that focuses on reducing uncertainty where it matters most (around potential optimal choices) is far more efficient.
  • Decoupling $u$ and $w$ Elicitation: The ability to elicit $u$ and $w$ independently is a significant advantage of the WEU model in an incremental setting, simplifying the optimization and query generation process compared to models where parameters are tightly coupled.
  • Guaranteed Performance: The maxmin approach provides a theoretical guarantee on the quality of the chosen lottery, bounding the potential loss incurred by stopping the elicitation early.

Key Results

  • The proposed incremental elicitation strategy (LA) significantly reduces the upper bound on the worst-case loss (UB($p^*, p^*$)) efficiently. For instance, it reaches a bound of 0.05 after only 15 queries on average.
  • The method is computationally efficient, with query times being very fast (e.g., less than 0.5 seconds for the 20th query in a large experiment).
  • Even when the elicitation is stopped early (based on the UB bound), the chosen lottery ($p^*$) is often near-optimal with respect to the true, hidden WEU model, as indicated by the much lower actual loss (LS($p^*$)).
  • The spline representation allows for a compact model of the DM's preferences even after a moderate number of queries, with $U_{min}/U_{max}$ and $W_{min}/W_{max}$ bounds showing that the functions are far from fully elicited but the estimated loss is small.

Limitations and Future Work

  • Spline Basis Choice: The paper uses cubic I-splines. While effective, the choice of the number of basis functions ($m$) and the subdivision points could influence performance. The paper uses $m=12$ for experiments.
  • Query Types: The paper focuses on specific query types (uQuery, wQuery). Exploring other query types or more sophisticated ways to choose the query parameter 'a' could further improve efficiency.
  • Simulated DM: The experiments rely on simulated DM responses. Real-world human behavior might introduce noise or inconsistencies not captured by the simulations.
  • Future Work: The authors suggest exploring more complex query generation strategies and investigating the impact of different spline bases or degrees. They also highlight that WEU's concurrent elicitation of $u$ and $w$ is a strong advantage over models like RDU.

Summary updated

BibTeX
@inproceedings{gilbert2017incremental,
  author = {Hugo Gilbert and Nawal Benabbou and Patrice Perny and Olivier Spanjaard and Paolo Viappiani},
  title = {{Incremental Decision Making Under Risk with the Weighted Expected Utility Model}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2017},
  doi = {10.24963/ijcai.2017/640},
}
Figure