Predicting Strategic Behavior from Free Text (Extended Abstract)

Omer Ben-Porat, Lital Kuchy, Sharon Hirsch, Guy Elad, Roi Reichart, Moshe Tennenholtz

Abstract

This research investigates whether unstructured natural language—specifically personal narratives—can predict individual behavior in strategic economic games. By extracting personality attributes from free text and employing a transductive clustering approach, the study successfully predicted moves in games such as "Chicken" and "Box." Notably, the text analyzed was authored before participants were aware of the games, suggesting that linguistic style contains latent signals of strategic decision-making.

The Strategic Signal in Natural Language

Traditionally, Natural Language Processing (NLP) and Economics have operated in separate spheres regarding the link between communication and action. NLP typically focuses on "non-strategic" actions, such as predicting consumer purchases from reviews. Conversely, Game Theory examines "strategic" actions—decisions where payoffs depend on the choices of others—but usually relies on highly structured, stylized communication (e.g., "I plan to choose Action A").

This paper addresses the gap: Can arbitrary free text, unrelated to a specific game, serve as a signal for strategic behavior? This is particularly relevant for the modern web, where users generate vast amounts of text that might reveal underlying behavioral tendencies.

Methodology: Transductive Attribute Clustering

The researchers developed a two-step computational pipeline:

  1. Attribute Extraction: Rather than using raw text or standard word-count features, the authors represented each text through "commonsensical personality attributes." Graduate students established a consensus on these attributes, which were then annotated via crowd-sourcing. This created a personality-based vector representation for each participant.
  2. Transductive Attribute Clustering (TAC): The authors utilized transductive learning, where the algorithm observes all input data (personality attributes) but only knows the game choices for a subset. A bottom-up agglomerative clustering algorithm (Ward’s minimum variance criterion) grouped participants by personality. Predictions for "unlabeled" participants were then based on the choices of others within the same cluster.

The authors argue that transductive learning is particularly suited for web applications, where large pools of unlabeled user text are often available before specific actions need to be predicted.

Experimental Design and Datasets

The study involved 271 participants recruited via Facebook. Each provided an English text of at least 1,000 characters describing a personal experience. Participants then played three one-shot, non-cooperative games:

  • Chicken: A coordination and conflict game (Speed vs. Stop).
  • Box: A congestion game involving choices between two boxes with different payoffs.
  • Door: A pure coordination game involving three doors.
Payoff matrices for the three games: (a) Chicken, (b) Box, and (c) Door.

The payoff structures for these games are illustrated in the figure above, highlighting the strategic interdependence of the players' choices. The dataset and code are available at https://github.com/omerbp/Predicting-NLPGT.

Results and Key Insights

The primary finding is that text provided by individuals unaware of the games still contained useful signals for predicting strategic choices. The TAC algorithm consistently outperformed the Majority Vote Classifier (MVC) baseline.

Distribution of participant choices across the Chicken, Box, and Door games, categorized by gender.

As shown in the distribution of choices, there is significant variation in how participants approached each game. Key insights include:

  • Personality as a Bridge: Personality attributes act as a robust intermediary between linguistic style and economic decision-making.
  • Ablation Success: The combination of clustering and personality-based representation was the "winning configuration" for Chicken and Box, and the second-best for Door.
  • Human Intuition: "Commonsensical" human-annotated attributes performed better than established automated tools like LIWC or IBM Personality Insights, suggesting that standard NLP tools may miss nuances relevant to strategic behavior.

Conclusion and Future Directions

While the results are promising, the authors note several limitations, including the relatively small sample size (N=271) and the reliance on human crowd-sourcing for attribute annotation. Future work aims to automate the extraction of these personality attributes and explore whether players can gain a strategic advantage by analyzing an opponent's text before a game. This research opens new avenues for understanding the intersection of language, personality, and strategic interaction on the modern web.

Summary updated

Abstract

The connection between messaging and action is fundamental both to web applications, such as web search and sentiment analysis, and to economics. However, while prominent online applications exploit messaging in natural (human) language in order to predict non-strategic action selection, the economics literature focuses on the connection between structured stylized messaging to strategic decisions in games and multi-agent encounters. This paper aims to connect these two strands of research, which we consider highly timely and important due to the vast online textual communication on the web. Particularly, we introduce the following question: can free text expressed in natural language serve for the prediction of action selection in an economic context, modeled as a game? We initiate research on this question by providing preliminary positive results.
BibTeX
@inproceedings{benporat2020predicting,
  author = {Omer Ben-Porat and Lital Kuchy and Sharon Hirsch and Guy Elad and Roi Reichart and Moshe Tennenholtz},
  title = {{Predicting Strategic Behavior from Free Text (Extended Abstract)}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/699},
}

Loading related content...

Figure