Level-0 Models for Predicting Human Behavior in Games

James R. Wright, Kevin Leyton-BrownJAIR 2019 · DOI · OpenReview

TL;DR

This paper tackles the challenge of accurately predicting human behavior in strategic games by moving beyond the assumption that "non-strategic" players (level-0) play randomly. They introduce and evaluate novel, data-driven models for level-0 behavior that significantly improve predictions from existing iterative game-theoretic models, demonstrating that non-strategic play is a crucial and complex aspect of human decision-making.

What Problem Does This Paper Address?

Traditional game theory often assumes players are perfectly rational and will play Nash equilibrium strategies. However, empirical studies consistently show that real human behavior deviates from these predictions. Behavioral game theory aims to build more realistic models. A prominent class of these models are "iterative reasoning" models, like Cognitive Hierarchy (CH) and Quantal Cognitive Hierarchy (QCH). These models assume players reason iteratively about others' reasoning, starting from a "level-0" baseline. The standard assumption for level-0 behavior is that players choose actions uniformly at random. This paper argues that this uniform assumption is often unrealistic and that a more sophisticated, data-driven approach to modeling level-0 behavior can dramatically improve predictive accuracy. This is crucial for applications like AI agents interacting with humans in strategic environments.

Methods

The core of this paper is the development and evaluation of new models for level-0 behavior. Instead of assuming uniform random play, they propose models that derive level-0 behavior from the game's structure using "features." These features are general rules that score each action based on aspects like potential payoffs (maxmax, minmax), fairness, and symmetry.

They explore two main ways to combine these features into a probability distribution over actions:

  1. Weighted Linear Specification: A linear combination of feature scores, normalized to produce probabilities.
  2. Logit Specification: A logit transformation of a weighted sum of feature scores, similar to discrete choice models.

They also investigate two feature transformations:

  1. Informativeness: Zeroes out features that are constant across all actions in a game, assuming they don't help distinguish choices.
  2. Normalized Activation: Scales feature values to the [0,1] range to prevent any single feature from dominating.

To find the best combination of features, functional forms, and transformations, they employ two model selection strategies:

  • Exhaustive Evaluation: For binary features, they systematically test all subsets with the linear functional form and informativeness transformation.
  • Bayesian Optimization (SMAC): For a broader search including real-valued features and different functional forms, they use SMAC to efficiently explore the model space.

Datasets

The study utilizes data from ten experimental studies on normal-form games. These datasets vary in size and experimental setup. The authors also construct a combined dataset, ALL10, which pools data from all studies for broader analysis. The paper notes that the data was preprocessed to normalize payoffs and ensure consistency.

Key Insights and Ideas

  • Non-uniform Level-0 is Crucial: The assumption that non-strategic players play randomly is a significant oversimplification. More sophisticated, game-structure-based models of level-0 behavior lead to substantially better predictions.
  • Fairness is a Powerful Predictor: The "maxmax fairness" feature consistently emerged as the most important predictor across various models, suggesting that human players are sensitive to the fairness of outcomes, even at a non-strategic level.
  • Real-Valued Features Enhance Models: While binary features provide a good baseline, incorporating real-valued versions of features (e.g., actual payoff values rather than just indicators) further boosts predictive accuracy.
  • Overfitting is a Real Risk: Simply adding more features or complexity can lead to overfitting, where models perform well on training data but poorly on unseen data. The study demonstrates this by showing test performance decreasing after a certain number of features.
  • Level-0 is Not Just a Starting Point for Higher Levels: The results suggest that a significant portion of players might actually be playing level-0 strategies directly, rather than just being the foundation for higher-level reasoning. The models estimated a substantial proportion of level-0 agents, sometimes over 50%.

Key Results

  • The proposed level-0 models, particularly the linear4 (using four binary features: maxmax payoff, maxmin payoff, maxmax fairness, max symmetric) and linear8 (augmenting linear4 with real-valued versions of its features), significantly improved prediction accuracy for iterative models like Poisson-QCH, Cognitive Hierarchy, and Level-k.
  • The linear8 model, with 9 parameters, achieved the best performance, estimating that over half of the agents were level-0.
  • The linear4 model, with 4 parameters, also provided substantial improvements and was a strong performer.
  • Adding more than four binary features to the linear model led to overfitting and decreased test performance.
  • The fairness[R] feature in the linear8 model was poorly identified, suggesting it might not be as predictive as initially thought, and omitting it (in the linear7 model) did not significantly harm performance.

Limitations and Future Work

  • Dataset Specificity: The study primarily uses two-player games. The authors believe their results would generalize to multi-player games due to the focus on agents not explicitly forming beliefs about others, but this remains an area for future work.
  • Feature Set Completeness: The authors acknowledge that their investigated features are not exhaustive and that other factors might influence non-strategic agents' actions.
  • Framing Effects: The paper briefly mentions that framing effects might play a role in determining level-0 behavior, suggesting this as a potential avenue for future research.
  • Interaction Effects: While the paper explores pairwise interactions, a deeper understanding of complex parameter interactions could further refine models.
  • Understanding Non-Strategic Behavior: Further research is needed to better capture the factors influencing non-strategic agents and to incorporate these into predictive models.

Summary updated

Abstract

Behavioral game theory seeks to describe the way actual people (as compared to idealized, "rational" agents) act in strategic situations. Our own recent work has identified iterative models, such as quantal cognitive hierarchy, as the state of the art for predicting human play in unrepeated, simultaneous-move games. Iterative models predict that agents reason iteratively about their opponents, building up from a specification of nonstrategic behavior called level-0. A modeler is in principle free to choose any description of level-0 behavior that makes sense for a given setting. However, in practice almost all existing work specifies this behavior as a uniform distribution over actions. In most games it is not plausible that even nonstrategic agents would choose an action uniformly at random, nor that other agents would expect them to do so. A more accurate model for level-0 behavior has the potential to dramatically improve predictions of human behavior, since a substantial fraction of agents may play level-0 strategies directly, and furthermore since iterative models ground all higher-level strategies in responses to the level-0 strategy. Our work considers models of the way in which level-0 agents construct a probability distribution over actions, given an arbitrary game. We considered a large space of alternatives and, in the end, recommend a model that achieved excellent performance across the board: a linear weighting of four binary features, each of which is general in the sense that it can be computed from any normal form game. Adding real-valued variants of the same four features yielded further improvements in performance, albeit with a corresponding increase in the number of parameters needing to be estimated. We evaluated the effects of combining these new level-0 models with several iterative models and observed large improvements in predictive accuracy.
BibTeX
@article{wright2019level,
  author = {James R. Wright and Kevin Leyton-Brown},
  title = {{Level-0 Models for Predicting Human Behavior in Games}},
  journal = {J. Artif. Intell. Res.},
  year = {2019},
  doi = {10.1613/jair.1.11361},
}
Figure