Abstract
Sender-receiver interactions, particularly persuasion games, are fundamental to economic modeling and artificial intelligence. Traditionally, these games rely on abstract or well-structured signals. This paper investigates the introduction of natural language into persuasion games through an online repeated interaction experiment. In this setup, an informed expert attempts to persuade an uninformed decision-maker (DM) to book a hotel by selecting specific reviews. Our study compares behavioral patterns between verbal and numerical communication, finding that natural language leads to more predictable and rational behavior. We evaluate several modeling approaches, including deep neural networks and linear classifiers, using both BERT-based and hand-crafted textual features. Our results indicate that sequential modeling combined with hand-crafted features achieves high accuracy in predicting DM decisions, offering new insights into the linguistic drivers of strategic decision-making.
Introduction
In traditional game theory, "persuasion games" involve an informed expert attempting to influence the actions of an uninformed decision-maker. Historically, these models have relied on numeric signals or structured data. However, real-world persuasion—from marketing to interpersonal negotiations—is conducted through natural language. This research addresses the gap between abstract economic models and the complexity of human language by analyzing how verbal communication influences strategic choices in a multi-round simulation.
Experimental Design: The Hotel Booking Game
To study these dynamics, we conducted an online experiment via Amazon Mechanical Turk involving 408 pairs of participants. The interaction was structured as a 10-round game between an "Expert" (acting as a travel agent) and a "Decision-Maker" (the client).
In each round:
- The Expert is presented with seven reviews and associated scores for a specific hotel, sourced from a dataset of 500,000 Booking.com reviews.
- The Expert selects exactly one review to show the Decision-Maker.
- The Decision-Maker observes only that single review and must decide whether to "Book" or "Stay Home."
- The Payoff: If the DM books, their payoff is a random draw from the hotel's actual score distribution (known only to the Expert). If they stay home, they receive a neutral default payoff.
This setup creates a strategic environment where the Expert must decide how much information to reveal to maximize bookings, while the DM must learn to interpret the Expert's signals over time.
Modeling and Feature Engineering
We compared several computational architectures to predict the Decision-Maker’s choices based on the history of the interaction:
- Support Vector Machines (SVM): Used as a non-sequential baseline to evaluate individual trials in isolation.
- Long Short-Term Memory (LSTM) Networks: Employed to capture the sequential nature of the 10-round game, accounting for how previous rounds influence current trust.
- Transformers: A sequence-to-sequence approach treating the game history as a translation-style task.
We also contrasted two types of textual representations:
- BERT-based Embeddings: Standard deep learning representations that capture broad linguistic context.
- Hand-Crafted Features (HCF): A set of 42 binary features identifying specific topics (e.g., mentions of the room or staff), sentiment intensity, and structural elements (e.g., the presence of a positive summary sentence).
Key Findings
1. Language Increases "Rationality"
One of the most striking observations was that verbal communication made human behavior more predictable and closer to a game-theoretic "pooling equilibrium" than numeric signals alone. In numeric versions of this game, players often struggle to find a stable strategy. In the verbal version, the nuances of language provided a more intuitive framework for establishing trust and strategic consistency.
2. The Superiority of Hand-Crafted Features
Contrary to the current trend favoring end-to-end deep learning, our Hand-Crafted Features (HCF) outperformed raw BERT embeddings. This suggests that in strategic contexts, high-level semantic concepts—such as specifically mentioning that there were "no negatives"—carry more weight than the general-purpose patterns learned by large language models. The best-performing model, a joint LSTM using HCF, achieved a per-trial prediction accuracy of 76.9%.
3. Sequentiality and Trust
Decision-makers do not evaluate reviews in a vacuum. Our models showed that sequentiality is key: a DM’s decision in round five is heavily influenced by whether the Expert provided an accurate or misleading review in rounds one through four. LSTMs significantly outperformed non-sequential models by accounting for this "memory" of the interaction.
4. Linguistic Nudges
Using SHAP (interpretability analysis), we identified the specific linguistic features that most strongly drove decisions. The most impactful features included:
- Explicit "nothing negative" statements.
- High-intensity positive descriptors.
- Mentions of the "room" within a negative context (which served as a strong deterrent).
Conclusion and Future Work
This research demonstrates that natural language is not just a medium for information, but a strategic tool that shapes the rationality of human agents in persuasion games. By combining sequential modeling with targeted linguistic features, we can accurately predict how humans will respond to persuasive text.
Future research will look toward moving from prediction to generation—developing AI agents capable of generating the most persuasive possible text based on the evolving state of a strategic interaction. Additionally, moving beyond the controlled environment of Mechanical Turk to high-stakes, real-world decision-making remains a vital next step.
Summary updated