On the Bayesian Rational Assumption in Information Design

Wei Tang, Chien-Ju HoHCOMP 2021 · OpenReview

TL;DR

This paper challenges the common assumption that people are perfectly rational decision-makers in information design scenarios. By developing and testing a new framework that incorporates more realistic human behavior (discrete choice and probability weighting), the authors show it better predicts real-world user actions and leads to more effective information design strategies.

What Problem Does This Paper Address?

Information design is about how an informed "sender" (like a recommender system or online retailer) can strategically reveal information to influence a "receiver" (a user) to make a desired decision. A prominent model for this is "Bayesian persuasion," which assumes the receiver is "Bayesian rational." This means the receiver perfectly updates their beliefs using Bayes' rule and then chooses the action that maximizes their expected utility.

However, real-world human behavior often deviates from this ideal. People aren't always perfectly rational; they make systematic errors in updating beliefs and in decision-making. This paper addresses the gap between the idealized assumptions of Bayesian persuasion and actual human behavior, aiming to create more effective information design strategies by accounting for these deviations.

Methods

The authors propose a new framework for information design that relaxes the Bayesian rational assumption. They incorporate two key components:

  • Discrete Choice Model: Instead of assuming a receiver deterministically chooses the action with the highest expected utility, this model introduces "noise" into the utility estimation. This accounts for inherent randomness in human decision-making and individual differences. The probability of choosing an action is then a smooth function of the receiver's utility, rather than a strict step function.
  • Probability Weighting: This component addresses deviations in belief updating. It models how humans might systematically over- or under-weight certain probabilities or the prior information, deviating from strict Bayesian updating.

The paper then reformulates the sender's optimization problem under this new framework and compares it to the standard Bayesian persuasion model.

Datasets

The paper does not use pre-existing datasets. Instead, it introduces a novel behavioral experiment conducted on Amazon Mechanical Turk with 400 participants. This experiment serves as the primary dataset for evaluating their proposed framework against the standard Bayesian rational model.

Key Insights and Ideas

  • Humans Aren't Always Bayesian Rational: The core insight is that the assumption of perfect Bayesian rationality is often too strong for modeling human behavior in information design. Empirical evidence consistently shows deviations.
  • Discrete Choice and Probability Weighting Capture Realism: The proposed framework, incorporating discrete choice and probability weighting, offers a more nuanced and realistic model of how humans process information and make decisions.
  • Better Predictions Lead to Better Design: By using a model that better reflects actual human behavior, information designers can create more effective strategies that achieve their goals more reliably.
  • Experimental Validation is Crucial: The paper emphasizes the importance of empirical validation. Their experiment provides strong evidence that their framework aligns better with real-world user behavior than the traditional Bayesian rational model.

Key Results

The paper presents both theoretical results and empirical findings:

  • Theoretical Framework: They formalize the information design problem under their new framework, showing it can be solved using optimization techniques.
  • Baseline Setting Comparison: In a simplified scenario with two states and two actions, they derive optimal information disclosure policies for both the Bayesian rational receiver and their proposed non-Bayesian rational receiver. The policies differ, highlighting the impact of the receiver model.
  • Experimental Findings:
    • Their experiment with 400 MTurk workers showed that participants' behavior significantly deviated from the Bayesian rational model.
  • The observed behavior was better explained by their proposed framework, which incorporates discrete choice and probability weighting, rather than a strict step function predicted by expected utility theory.
  • Quantitative evaluation using cross-validation showed their model achieved significantly lower prediction errors (sum of squared residuals) compared to the Bayesian rational model (e.g., 0.0417 vs. 0.1231 for the high prior treatment).
  • Figure 4 demonstrates that their model's predicted sender utility aligns much better with the empirical sender utility collected from the experiment than the utility predicted by the Bayesian rational model.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Experimental Setup Abstraction: The experiment uses a simplified "urn and ball" scenario. Future work could explore more realistic experimental setups that directly mimic real-world scenarios (e.g., online retail decisions).
  • Information Presentation Simplicity: The experiment's information presentation (ball compositions) was kept relatively simple. Future research should investigate how different ways of presenting information affect findings and the generalizability of their results.
  • Model Complexity: While their framework is more realistic, it introduces more parameters (e.g., for probability weighting). Exploring alternative behavioral models beyond discrete choice and probability weighting is also suggested.
  • Societal Impacts: The paper raises concerns about the potential negative societal impacts of information design if the sender's objectives (e.g., profit maximization) are not aligned with societal welfare. They suggest considering these impacts and potential regulations.
  • Algorithmic Solutions: For general information design problems, computing optimal structures can be computationally hard (#P-hard). Future work could explore whether their framework maintains or changes these complexity results and if polynomial-time algorithms can be developed.

Summary updated

BibTeX
@inproceedings{tang2021bayesian,
  author = {Wei Tang and Chien-Ju Ho},
  title = {{On the Bayesian Rational Assumption in Information Design}},
  booktitle = {HCOMP},
  year = {2021},
  url = {https://openreview.net/forum?id=69w0McNdaL},
}
Figure