Three-dimensional attention Transformer for state evaluation in real-time strategy games

Yanqing Ye, Weilong Yang, Kai Qiu, Jie ZhangarXiv 2025 · DOI · arXiv

Abstract

State evaluation in Real-Time Strategy (RTS) games is a complex task due to vast state spaces and intricate dependencies. This paper presents the TSTF Transformer (Tri-dimensional Space-Time-Feature), which enhances standard spatiotemporal modeling with a specialized feature attention layer. Experimental results on the microRTS benchmark demonstrate that the TSTF architecture significantly outperforms state-of-the-art models like TimeSformer, particularly in the early game, while utilizing 14% fewer parameters.

Introduction

In RTS games such as StarCraft or microRTS, determining which player holds the advantage—known as state evaluation—is a fundamental challenge. Unlike board games with perfect information and discrete moves, RTS environments feature hidden information, continuous-time dynamics, and complex spatial dependencies. Traditional evaluation functions often rely on simple material counts, which fail to capture tactical nuances like positioning or unit health distributions.

A typical game state in the microRTS environment, illustrating the grid-based spatial layout and various unit types.

Methodology: The TSTF Architecture

The core innovation of this work is the TSTF Transformer, which decomposes the attention mechanism into three cascaded modules to handle the multi-dimensional nature of game data:

  1. Spatial Attention (SA): Captures relationships between different map coordinates within a single time step.
  2. Temporal Attention (TA): Models the evolution of the battlefield across 500 discrete time steps, capturing long-range strategic shifts.
  3. Feature Attention (FA): Explicitly models associations between different data "planes" (e.g., unit type, health, ownership). This prevents the loss of logical interdependencies that occurs when features are flattened into a single vector.

The model processes a 16x16 grid with 5 primary feature dimensions using 4x4 non-overlapping patches and a specialized "situation token" for final win/loss prediction.

Experimental Setup

The researchers utilized the microRTS platform, generating a dataset from a round-robin tournament between 10 diverse AI strategies. The dataset comprises 3,150 matches played on a 16x16 map, partitioned into training (1,800), testing (900), and validation (450) sets.

Key Results and Discussion

The TSTF Transformer demonstrates significant improvements in both efficiency and predictive power. By treating feature dimensions as a separate axis for attention, the model more effectively learns the significance of specific unit attributes.

Performance metrics across game progress, showing TSTF's superior accuracy and stability compared to TimeSformer and traditional evaluation methods.

Performance Metrics

  • Early-Game Accuracy: The 8-layer TSTF Transformer achieved 58.7% accuracy in the first 4% of game progress, compared to just 41.8% for the 12-layer TimeSformer.
  • Mid-Game Convergence: Accuracy reached 97.6% by the 40% mark of game progress.
  • Efficiency: The TSTF-8 model reduced parameter count from 5.54M to 4.75M relative to the baseline.
  • Training Speed: TSTF models reached high accuracy within the first two training epochs, indicating faster convergence than traditional architectures.

Conclusion

The TSTF Transformer's tri-dimensional approach provides a more robust framework for analyzing "thick" game data where features carry distinct logical weight. While currently tested on 16x16 maps, future work will focus on scaling the architecture to larger environments and exploring its applicability in other domains requiring multi-dimensional temporal analysis, such as autonomous driving.

Summary updated

BibTeX
@misc{ye2025three,
  author = {Yanqing Ye and Weilong Yang and Kai Qiu and Jie Zhang},
  title = {{Three-dimensional attention Transformer for state evaluation in real-time strategy games}},
  year = {2025},
  eprint = {2501.03832},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.03832},
}
Figure