TL;DR
This paper presents a scoping review of 76 research papers exploring the application of Large Language Models (LLMs) in video games, published between 2022 and early 2024. It categorizes this burgeoning research into four key themes: Game AI and Agents, Game Development and Play, Narrative/Story/Dialogue, and Game Research and Reviews, offering a crucial snapshot of the current landscape and a foundation for future work.
What Problem Does This Paper Address?
The rapid emergence of powerful LLMs like ChatGPT has sparked immense interest in their potential applications across various domains, including video games. While generative AI has a long history in games, LLMs offer unprecedented capabilities for language processing and generation, promising to revolutionize game design, development, and research. However, the sheer volume and speed of new research make it challenging to grasp the current state of LLM integration in games. This paper addresses this by providing a structured overview and preliminary scoping review of existing research, aiming to map out the key areas of investigation, identify common themes, and highlight emerging trends.
Methods
The authors conducted a scoping review by performing a Google Scholar search for papers published between 2020 and early 2024, using a specific search string combining terms for "LLM" and "video game." They reviewed the first 500 search results, then analyzed the titles, abstracts, and full text of relevant papers to identify those reporting original research on LLMs and video games. The identified papers were then coded and categorized into four main themes: Game AI and Agents, Game Development and Play, Narrative, Story, and Dialogue, and Game Research and Reviews. An additional category for recommendation research using game datasets was also noted. The review then involved a detailed analysis of the papers within each theme, summarizing their focus, sub-themes, and findings.
Datasets
The paper does not explicitly list specific datasets used for training or evaluation within the reviewed papers, beyond mentioning that some papers utilized game data sets for recommendation research. It notes that 9 such papers were identified but excluded from the primary focus of the review.
Key Insights and Ideas
- LLMs are rapidly transforming game research: The sheer volume of papers published in 2023 alone underscores the explosive growth of LLM research in the gaming domain.
- GPT is the dominant LLM: The vast majority of reviewed papers (85.5%) utilized GPT models, highlighting their current prevalence in this research area.
- Broad application across game development: LLMs are being explored for a wide range of applications, from generating game content and mechanics to enhancing AI agents and facilitating human-AI collaboration.
- Narrative and dialogue generation shows promise: LLMs are proving adept at generating dialogue and stories, though challenges remain in ensuring coherence and avoiding bias.
- LLMs as tools for researchers: Beyond game creation, LLMs are being used to analyze game reviews, generate synthetic data, and even assist in research processes.
- Human interpretability and social interaction are key benefits: Researchers highlight LLMs' ability to provide interpretable reasoning and foster more believable social behaviors in game agents.
- Challenges remain in logical reasoning and consistency: Despite their advancements, LLMs still struggle with complex logical reasoning and can exhibit unpredictability, particularly in live game environments.
Key Results
The review identified 76 relevant papers, with the majority (90.7%) published in 2023. The papers were distributed across four main themes:
- Game AI and Agents: 35.5% of papers focused on agent design, reinforcement learning, and collaboration.
- Game Development and Play: 32.9% of papers explored content generation, serious games, game design, and RPGs.
- Narrative, Story, and Dialogue: 22.4% of papers dealt with dialogue, story, and quest generation.
- Game Research and Reviews: 9.2% of papers focused on analyzing game data and reviews.
Commonly reported positive highlights included human interpretability, the creation of social behaviors and experiences, foundational AI skills (like playing games), and empowering non-developers. Negative aspects noted were a lack of logical reasoning ability and unpredictability in live game contexts.
Limitations and Future Work
The authors acknowledge that this is a "preliminary scoping review" and that the field is rapidly evolving, making it challenging to capture a complete picture. They explicitly excluded research that, while potentially applicable to games, had not yet been applied. They also did not explicitly search for areas like gamification or other generative models beyond LLMs.
Future work is implicitly suggested by the identified research gaps and challenges. The authors emphasize the need for continued research to keep pace with the rapid advancements in LLMs and their application to games. They anticipate that capturing the state of the field will be an ongoing challenge, but express excitement about the future directions LLMs will enable in games research and development.
Summary updated