Abstract
Defining and measuring decision-making styles, also known as playstyles, is crucial in gaming, where these styles reflect a broad spectrum of individuality and diversity. However, finding a universally applicable measure for these styles poses a challenge. Building on Playstyle Distance, the first unsupervised metric to measure playstyle similarity based on game screens and raw actions, we introduce three enhancements to increase accuracy: multiscale analysis with varied state granularity, a perceptual kernel rooted in psychology, and the utilization of the intersection-over-union method for efficient evaluation. These innovations not only advance measurement precision but also offer insights into human cognition of similarity. Across two racing games and seven Atari games, our techniques significantly improve the precision of zero-shot playstyle classification, achieving an accuracy exceeding 90 percent with fewer than 512 observation-action pairs, which is less than half an episode of these games. Furthermore, our experiments with 2048 and Go demonstrate the potential of discrete playstyle measures in puzzle and board games. We also develop an algorithm for assessing decision-making diversity using these measures. Our findings improve the measurement of end-to-end game analysis and the evolution of artificial intelligence for diverse playstyles.
Introduction
In both human and artificial intelligence, "how" an agent plays is often as significant as "how well" it performs. Quantifying these decision-making styles, or playstyles, is essential for creating diverse AI behaviors and understanding player individuality. However, playstyle is notoriously difficult to measure objectively. Most existing approaches rely on hand-crafted features—such as "kill-death ratios" or "resource collection rates"—or supervised labels, which are domain-specific and expensive to produce.
This research builds upon the concept of "Playstyle Distance," an unsupervised metric that discretizes game states to compare policies. Previous iterations of this method faced a "Goldilocks" problem: if state representation is too broad, nuance is lost; if it is too fine, the agent rarely visits the same state twice, making comparison impossible. This paper introduces a robust framework to overcome these limitations using perceptual similarity.
Methodology
The authors utilize the Hierarchical State Discretization (HSD) framework, employing a Vector Quantized Variational Autoencoder (VQ-VAE) to transform continuous game screens into discrete "state codes." To refine this process, three major enhancements were introduced:
- Multiscale Analysis: Rather than selecting a single level of state granularity, the model utilizes a mixture of scales. This approach emulates human judgment, which simultaneously processes "big picture" context and "fine detail" nuances.
- Perceptual Kernel: Drawing from the Weber-Fechner Law in psychophysics, the researchers converted raw policy distances into a probabilistic similarity score using an exponential kernel: $P(d) = e^{-d}$. This reflects the "Magnitude Effect," where humans are highly sensitive to small differences between similar items but less sensitive to differences between items that are already distinct.
- Intersection-over-Union (Jaccard Index): By incorporating the ratio of shared states to total visited states, the metric can distinguish between styles that share local action patterns but navigate to entirely different regions of the game world.
- Diversity Assessment: An iterative algorithm was developed to count "diverse" trajectories by determining if a new gameplay session is sufficiently similar to a library of previously recorded sessions.
Experimental Setup
The metric was validated across a wide array of environments to test its generalizability:
- TORCS & RGSK: A racing simulator with 25 rule-based AI styles and a human dataset featuring 24 players following specific behavioral instructions (e.g., "driving on grass").
- Atari 2600: Seven classic games (including Asterix and Breakout) featuring 20 different Deep Reinforcement Learning (DRL) agents per game.
- Puzzle and Board Games: 2048 was used to test high-entropy state matching, while a dataset of 200 human players from the Fox Go server (ranging from 1 Dan to 9 Dan) tested the metric's ability to identify individual human signatures.
Key Insights and Results
High-Precision Zero-Shot Classification
The multiscale approach achieved over 90% accuracy in classifying playstyles across racing and Atari environments without any style-specific training. By fusing distance metrics with the Jaccard Index, the model effectively removed the need for manual "sample threshold" tuning, a significant hurdle in previous unsupervised methods.
Extreme Data Efficiency
One of the most striking findings is the metric's efficiency. The system can identify a player's style with high confidence using only 512 observation-action pairs. In most modern games, this equates to roughly 10 to 20 seconds of gameplay, or less than half of a standard game episode.
Human Identification in Go
The framework successfully identified specific human players from a pool of 200. Remarkably, the system could achieve up to 97% identification accuracy given enough samples, even when analyzing only the opening 10 moves of a match.
Measuring AI Diversity
By applying the diversity algorithm to DRL agents, the researchers demonstrated that certain architectures, such as Implicit Quantile Networks (IQN), naturally produce more diverse behaviors than standard Deep Q-Networks (DQN), even when trained on identical tasks.
Discussion and Limitations
While the results are promising, the method's success is inherently tied to the quality of the underlying state encoder (the VQ-VAE). If the encoder fails to capture a visually subtle but strategically vital feature, the playstyle measure will overlook it. Furthermore, in high-randomness environments like 2048, where states are rarely revisited, the Jaccard Index proved less effective. In such cases, local action distributions remain a more reliable signal than state-visiting distributions.
Conclusion
This research advances the measurement of end-to-end game analysis by shifting the focus from raw distance to perceptual similarity. The ability to identify playstyles rapidly and without supervision opens new doors for game recommendation systems—suggesting titles based on a player's inherent decision-making style—and for the development of LLM-based agents that exhibit consistent, diverse personalities.
Summary updated