Introduction
In behavioral economics, human decision-making is frequently characterized as "predictably irrational." We often deviate from the utility-maximizing behavior predicted by classical economic models, falling prey to marketing tactics and context-dependent shifts in preference. However, a recent paper presented at AAAI-21, "Apparently Irrational Choice as Optimal Sequential Decision Making," proposes a shift in perspective. It suggests that these "biases" are not flaws, but rather the hallmarks of a computationally rational system designed to make high-quality decisions under constraints of time, noise, and cognitive cost.
The Problem: Context Effects and "Irrationality"
Psychologists have long documented context effects, where the introduction of a third, seemingly irrelevant option (C) alters the preference between two original options (A and B). Two prominent examples include:
- The Attraction Effect: A preference for a target option increases when a "decoy"—an option clearly inferior to the target—is introduced.
- The Compromise Effect: A tendency to choose the "middle" option to avoid extremes.
Standard normative theories view these shifts as irrational, as the intrinsic value of an apple should not change based on the existence of a poor-quality cake. This research seeks to explain why such behavior might be evolutionarily advantageous by framing choice as a problem of computational rationality.
Methodology: Choice as a POMDP
The authors modeled the decision-making process as a Partially Observable Markov Decision Process (POMDP), solved using a Deep Reinforcement Learning agent (ACER algorithm).
The agent is equipped with specific "cognitive actions":
- Calculation: Estimating the absolute value of an individual option.
- Comparison: Directly comparing two options to determine which is superior.
- Choice: Finalizing the decision and terminating the process.
Crucially, every cognitive action incurs a computational cost, and all observations are noisy. The agent's objective is to maximize the expected value of the chosen item while minimizing the total cost of deliberation. The model was trained on "Ecological Distributions"—simulated choice sets reflecting real-world value distributions from landmark behavioral studies (e.g., Wedell 1991, Trueblood 2012).
Key Results: Replicating Human Biases
The "Integrated Agent" successfully replicated the three major context effects—Attraction, Compromise, and Similarity—using a single, fixed policy.
Key insights from the model include:
- Efficiency of Comparison: The agent discovered that direct comparison is often more reward-efficient than absolute calculation, especially for similar items.
- Decoys as Information Landmarks: The attraction effect emerges because a dominated decoy provides a low-cost reference point, reducing uncertainty about the target option.
- Emergent Bias: The agent was not programmed to be biased; rather, the optimal policy for a noisy, cost-constrained environment naturally exhibits context sensitivity.
- Sensitivity to Noise and Cost: As the cost of thinking increases, context effects weaken as the agent stops gathering information earlier. Conversely, higher calculation noise leads to a greater reliance on comparisons, strengthening these effects.
Conclusion
This research suggests that "irrational" human choices are actually the result of a highly efficient sequential decision-making process. By balancing the accuracy of a choice against the computational cost of making it, our brains use clever shortcuts to navigate a complex world. What looks like a bias from a distance is, in fact, an optimal strategy for a system where thinking is never free.
Summary updated