Researchers from University College London have developed an "Immersive Text Game" that bridges the gap between interactive entertainment and psychological profiling. By integrating large language models with commonsense reasoning and personality classification, the system allows players to roleplay within famous TV scripts while simultaneously analyzing their Myers-Briggs Type Indicator (MBTI) personality based on their dialogue.
Addressing the Limitations of Text Adventures
Traditional text adventure games often struggle with a binary choice: limited, pre-defined "choose-your-own-adventure" paths or open-ended AI generation that lacks narrative coherence. This project aims to create a game that is both narratively constrained—staying within the worlds of Friends or Sherlock—and analytically capable of evaluating the player’s personality through natural language input.
System Architecture and Methods
The authors built a multi-layered pipeline to manage the complexities of dialogue and logic.
The core engine utilizes several key components:
- Controlled Generation (PPLM + GPT-2): To prevent the model from wandering off-topic, the researchers employed the Plug and Play Language Model (PPLM). This "Bag-of-Words" approach steers GPT-2 toward specific topics or storylines without the need for expensive retraining.
- Commonsense Reasoning (COMET): To maintain narrative flow, the system integrates COmmon-mense Transformers (COMET). This provides "if-then" logic to help the AI understand character intent and reactions.
- Keyword Extraction: Relevant terms are pulled from Wikipedia summaries of the TV shows to drive the plot forward within the PPLM framework.
- Personality Classification (BERT): A BERT model, fine-tuned on a dataset of social media posts labeled with MBTI types, aggregates player inputs to predict their personality type.
Datasets and Personality Profiling
The system was trained and tested using scripts from Friends and Sherlock. For the personality layer, the researchers used a dataset of 8,675 samples, each containing 50 social media posts labeled with self-reported MBTI types.
A significant challenge identified during development was the inherent imbalance in the personality dataset. As shown in the distribution of types, certain categories like INFP are much more prevalent than others like ESTJ, which impacts the model's ability to generalize across all 16 types.
Key Results and Insights
The BERT-based classification model achieved an accuracy of approximately 0.70 across most categories, significantly outperforming baseline models like Multinomial Naive Bayes (0.5999) and XGBoost (0.6553). The model was particularly effective at identifying ESFJ (0.7536) and ISTJ (0.7429) types.
Qualitatively, the system successfully mimicked script-style dialogue, including stage directions. The researchers noted that "playing as a character" often reveals a player's true traits, as linguistic patterns like word choice and syntax persist even during roleplay.
Limitations and Future Directions
Despite its successes, the system faces several hurdles:
- Data Imbalance: The accuracy for underrepresented types, such as ENFJ, was as low as 0.3074 due to the lack of training samples.
- Input Length: BERT performs best on long texts. Short game inputs (e.g., "Fine.") often caused the model to default to an "ISTJ" (serious/quiet) classification.
- Latency: The multi-model pipeline introduces noticeable waiting times for text generation.
Future work may involve transitioning to the "Big Five" personality model. Unlike the 16 discrete MBTI categories, the Big Five uses a 5-dimension score vector, which may offer a more nuanced and accurate psychological profile for interactive applications.
Summary updated