TL;DR
This paper explores whether free-text personal narratives can predict an individual's strategic behavior in economic games, even when the text is unrelated to the game itself. They developed a novel approach using crowd-sourced "commonsensical" personality attributes derived from text to train a transductive clustering model, demonstrating that personality traits inferred from casual writing can indeed forecast game-theoretic decisions.
What Problem Does This Paper Address?
The paper bridges two research areas: online applications that use text for action prediction (like web search and sentiment analysis) and economics literature that studies how structured communication influences strategic decisions in games. The core question is whether unstructured, free-text personal narratives, written without knowledge of an upcoming game, can serve as a signal for predicting an individual's actions in an economic game context. This is challenging because traditional economic models often rely on explicit communication or observed behavior within the game, whereas this work aims to leverage pre-existing, unrelated personal text.
Methods
The authors employ a two-step approach:
- Attribute Collection: Instead of relying on existing NLP tools, they developed a set of "commonsensical" personality attributes (e.g., adventurous, kind, rational). Two graduate students initially identified 24 such attributes. These attributes were then assigned numerical scores (0-5) to participant texts via crowd-sourcing. This human-driven annotation process was chosen to avoid the noise of automatic extraction and to focus on intuitive personality descriptors.
- Transductive Clustering: A transductive learning approach is used. This means the model is trained on a set of texts with known personality attributes and a subset of known game actions, along with a set of texts with known attributes but unknown actions. The goal is to predict the actions for the unlabeled texts. Specifically, they use a deterministic clustering algorithm (TAC) applied to the attribute-based representation of participants. This clustering is performed without access to the game choices, allowing a single clustering to be used for predicting actions across multiple games. The algorithm then predicts an unlabeled example's action based on a majority vote within its cluster from the labeled examples.
Datasets
The primary dataset consists of:
- Personal Texts: 271 personal texts written by university students, each at least 1000 characters long, on a topic of their choice.
- Game Play Data: The same 271 participants played three one-shot games: Chicken, Box, and Door. Their choices in these games serve as the labels for prediction.
No pre-existing benchmarks were directly used for training; instead, the authors created their own dataset and methodology.
Key Insights and Ideas
- Personality as a Predictor: The core insight is that personality traits, as inferred from casual, unrelated personal writing, can be predictive of strategic behavior in economic games. This suggests a deeper connection between personality and decision-making than previously explored in this context.
- "Commonsensical" Attributes: The success of their custom-defined, intuitive personality attributes, derived from human judgment rather than complex NLP models, is a significant finding. This highlights the potential of human-curated features in complex prediction tasks.
- Transductive Learning Advantage: The transductive learning setup is well-suited for this problem, as it allows leveraging all available text data for training, which is realistic for systems that might process large volumes of text before specific predictions are needed.
- Clustering for Multi-Game Prediction: Using a single clustering algorithm on attribute representations to predict actions across multiple different games is an efficient and novel approach.
Key Results
- TAC Outperforms Baselines: The proposed Transductive Attribute Clustering (TAC) algorithm generally outperforms the Majority Vote Classifier (MVC) on Macro Average F1 and Macro Weighted Average F1 scores, indicating better performance across both majority and minority classes.
- Competitive Accuracy: While MVC sometimes edges out TAC on Accuracy (especially for the Door game), TAC's performance is competitive and demonstrates a better balance across different evaluation metrics.
- Attribute Importance: Ablation studies suggest that the clustering step and the personality attribute representation are crucial for TAC's performance.
- Custom Attributes Shine: In comparisons against models using established NLP tools like IBM Personality Insights and LIWC, TAC with its custom attributes often performs comparably or better, especially when combined with other attributes. The authors' custom attribute set showed additive benefits when combined with IBM attributes.
Limitations and Future Work
- Data Size and Bias: The dataset, while large for controlled experiments, is still considered small for heavy inductive methods like deep neural networks. Biases in gender and age were introduced due to the university-based participant pool.
- Attribute Subjectivity: The extraction of personality attributes relied on human judgment, which is inherently subjective. While crowd-sourcing quality control was implemented, potential for noise remains.
- Automatic Attribute Extraction: A significant future direction is to develop automatic classifiers for extracting these commonsensical attributes from text, enabling a fully automated system.
- Larger Datasets and Inductive Methods: Generating larger datasets would allow for the application of more complex inductive methods and further validation of the approach.
- Strategic Text Generation: Future work could explore scenarios where agents are aware of the game and might write texts strategically, moving closer to the "cheap talk" literature.
Summary updated