TL;DR
This paper introduces TAG, a Java-based framework designed to simplify the development and benchmarking of AI agents for modern tabletop games. By providing a common API, reusable components, and a flexible structure, TAG aims to accelerate research in General Game AI, enabling easier comparison and development of agents across a diverse range of games.
What Problem Does This Paper Address?
Developing AI for complex games, especially tabletop games, presents several challenges. These games often involve intricate rules, partial observability (hidden information), stochasticity (randomness), and a wide variety of game mechanics. Creating AI for each new game from scratch is time-consuming and makes it difficult to compare the performance of different AI algorithms across a standardized set of games. Existing frameworks might be too specialized or lack the flexibility to accommodate the diverse nature of tabletop games. TAG aims to bridge this gap by offering a unified platform that abstracts away common game elements and provides a consistent interface for AI agents.
Methods
TAG is built around several core concepts:
- Common API for AI Agents: A standardized interface allows AI agents to interact with any game implemented within TAG, promoting reusability and comparability.
- Reusable Components: TAG provides pre-defined components like tokens, dice, cards, counters, and various board types (grid, graph). These can be easily extended or used directly to build new games.
- Flexible Game Structure: Games are defined by a
GameState(representing the current state of the game) and aForwardModel(handling game logic, state transitions, and rule enforcement). This separation allows for modular game development. - JSON Import: Game data, such as card definitions or board layouts, can be imported using JSON, simplifying content creation.
- Logging and Analysis: The framework includes logging capabilities to collect detailed game statistics (action space, branching factor, hidden information, etc.) for in-depth analysis of AI performance.
- Modular Packages: TAG is organized into packages like
core,games,gui, andplayers, promoting a clean and maintainable architecture.
Datasets
The paper does not explicitly mention training datasets. However, it details the implementation of seven tabletop games within the framework:
- Tic-Tac-Toe
- Love Letter
- Colt Express
- Uno
- Virus!
- Exploding Kittens
- Pandemic
These games serve as benchmarks and examples for developing new games within TAG. The framework also supports loading game-specific data via JSON files.
Key Insights and Ideas
- Abstraction is Key: The success of TAG hinges on its ability to abstract common tabletop game mechanics into reusable components and a consistent API. This significantly reduces the boilerplate code required to implement new games.
- Partial Observability is a Major Challenge: Many of the implemented games feature partial observability, mirroring real-world tabletop games. This necessitates AI agents that can reason under uncertainty and build internal "belief systems."
- Game Speed is Surprisingly High: Despite the complexity of some games, the framework is highly optimized, allowing for millions of function calls per second, which is crucial for running extensive AI evaluations.
- Diversity of Games: The selection of games, ranging from simple (Tic-Tac-Toe) to complex with significant hidden information (Pandemic, Uno), showcases the framework's versatility.
- Focus on General Game AI: The ultimate goal is to facilitate research into AI that can generalize across a wide variety of games, rather than excelling at just one.
Key Results
The paper presents an analysis of the implemented games using random agents, measuring various metrics:
- Action Space Size ($\mu_1$) and Branching Factor ($\mu_2$): These metrics highlight the complexity of decision-making. For instance, Pandemic has a high action space size (11.15) and branching factor (17.97), indicating significant complexity.
- State Size ($\mu_3$) and Hidden Information ($\mu_4$): Pandemic has the largest state size (138.00), while Uno exhibits high hidden information (92.3%).
- Game Speed ($\mu_5$): Most games execute very quickly, with functions like
nextandactionsreaching over 1 million calls per second. - Game Length ($\mu_6$): Game lengths vary significantly, from a few ticks for Tic-Tac-Toe to hundreds for Uno.
- Reward Sparsity ($\mu_7$): Exploding Kittens shows a wide spread of rewards, suggesting interesting challenges for reward-based learning algorithms.
The analysis demonstrates that TAG can effectively capture and quantify the complexity of different tabletop games, providing valuable insights for AI research.
Limitations and Future Work
- Limitations: The paper acknowledges that while the framework is robust, some games might require custom extensions to fully leverage their unique mechanics. The current implementation of tournaments is limited to round-robin.
- Future Work: The authors envision expanding TAG to include more complex games like "Betrayal at House on the Hill" (which features cooperative and competitive phases), exploring knowledge transfer between games, developing AI that can adapt to changing game rules (like in Fluxx), and integrating procedural content generation for game assets. Further research directions include parameter optimization, building belief systems for handling hidden information, and supporting vector and feature-based observations for advanced AI integration.
Summary updated