An unsupervised video game playstyle metric via state discretization

Chiu-Chou Lin, Wei-Chen Chiu, I-Chen Wu

Abstract

On playing video games, different players usually have their own playstyles. Recently, there have been great improvements for the video game AIs on the playing strength. However, past research for quantifying playstyles often relied on manually defined features. This paper introduces the first unsupervised metric to quantify video game playstyles directly from raw game observations and actions using a novel Hierarchical State Discretization (HSD) approach.

Introduction

In the world of gaming, "playstyle" is a familiar but slippery concept. We know a "speedrunner" looks different from a "completionist," and a "cautious driver" differs from an "aggressive" one. However, quantifying these differences usually requires "heuristic rules"—manually defined features like average speed, position on a track, or frequency of button presses. This manual approach is brittle; it doesn't scale well to new games and fails to capture the subtle nuances of modern Deep Reinforcement Learning (DRL) agents. The challenge addressed here is: how do we measure the "distance" between two playstyles using only raw pixels and actions, without knowing anything about the game's internal logic?

Methodology: Hierarchical State Discretization

The authors propose a framework that treats playstyle as a mapping from game states to action distributions. The core technical hurdle is that raw game screens (high-dimensional pixels) are almost never identical, making it hard to compare what two players did "in the same situation."

To solve this, the researchers use a modified Vector Quantised-Variational AutoEncoder (VQ-VAE). While standard VQ-VAEs often produce too many states for effective comparison, the proposed Hierarchical State Discretization (HSD) uses a multi-hierarchy stacked structure to further compress these representations into a manageable number of latent discrete states. Crucially, the model includes a "policy decoder" that forces the discrete states to retain information relevant to the actions the player took.

The Hierarchical State Discretization (HSD) architecture, showing the multi-level encoding and action-aware policy decoder.

Once the game is discretized into states ($s$), the metric looks at the intersection of states visited by two different agents. For every shared state, it compares the distribution of actions taken by each agent using the 2-Wasserstein distance. The final "playstyle distance" is the average of these action-distribution differences across all shared states.

Experimental Results

The researchers evaluated their metric across three distinct environments: TORCS (a 3D racing simulator), RGSK (a racing game with human players), and Atari 2600.

Performance in TORCS

In the TORCS environment, the HSD metric was tested against 25 distinct styles. It achieved 69.73% accuracy in identifying the specific style an agent was using, significantly outperforming raw pixel or downsampling methods which achieved only 4% accuracy.

Performance comparison of HSD against Pixel and LRD baselines across different style sets in TORCS.

Human Style Detection and Atari Generalization

In the RGSK dataset, the metric reached over 93% accuracy in distinguishing between human playstyles, proving it works on the messy, inconsistent data generated by real people. Furthermore, the metric successfully differentiated between different RL algorithms (such as DQN, Rainbow, and IQN) in 5 out of 7 Atari games, often outperforming baselines even in complex environments like Space Invaders and Seaquest.

Conclusion

This work demonstrates that discretization acts as a functional bridge, grouping "meaningfully similar" situations together without requiring manual feature engineering. While the current model assumes a single consistent style per dataset, future work could explore measuring styles based on segments of trajectories to capture temporal dynamics and style shifts. By focusing on the "intersected states" where players face the same latent challenges, HSD provides a robust, unsupervised path toward understanding player behavior in complex digital environments.

Summary updated

Abstract

On playing video games, different players usually have their own playstyles. Recently, there have been great improvements for the video game AIs on the playing strength. However, past researches fo...
BibTeX
@inproceedings{lin2021unsupervised,
  author = {Chiu-Chou Lin and Wei-Chen Chiu and I-Chen Wu},
  title = {{An unsupervised video game playstyle metric via state discretization}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=SElwunUD8e5},
}
Figure