Abstract
Artificial Intelligence in Real-Time Strategy (RTS) games like StarCraft II is hindered by the computational cost of processing sparse game maps as dense images. This paper introduces a biologically inspired, unsupervised Hebbian learning method that treats game units as sparse point sets. By employing a novel "neuron activity aware" learning rule and k-Winner-Takes-All (k-WTA) competition, the model achieves superior prediction accuracy for unit movements. Notably, the approach requires 110x fewer activations and 793x fewer FLOPs than traditional frame-based models, while demonstrating significantly better data efficiency.
Introduction
StarCraft II maps are inherently sparse. In a typical 256x256 resolution map, even a maximum army occupies only about 0.6% of the total pixels. Processing these vast empty areas using standard Convolutional Neural Networks (CNNs) is computationally wasteful.
The authors propose that representing units as a point set—a collection of (x, y) coordinates—is a far more efficient way to model game dynamics. However, learning meaningful representations from such sets without explicit supervision remains a challenge.
Methodology
The researchers developed an Encoder-LSTM-Decoder architecture. The core innovation lies in the unsupervised training of the encoder, which utilizes principles from biological neural networks.
Hebbian Learning on Point Sets
Instead of standard backpropagation, the encoder uses a variant of Grossberg’s "instar rule." In this unsupervised method, weights are updated based on the proximity of input points to weight vectors. To ensure sparse and distinct representations, a k-Winner-Takes-All (k-WTA) mechanism is used, where only the 'k' most active neurons fire for a given input.
Neuron Activity Aware Learning
Hebbian learning often suffers from "dead" neurons or over-active neurons that dominate the representation. To counter this, the authors derived an optimal neuron activity level ($p^* = k/d$) using Lagrange multipliers. They introduced anti-Hebbian learning to push the weights of over-active neurons away from data points, while standard Hebbian learning pulls under-active neurons toward them.
The resulting latent vectors are fed into an LSTM to model temporal changes and a Decoder to reconstruct the predicted future positions of units.
Experimental Results
The model was trained on StarCraft II replay data (Terran vs. Terran) extracted via the PySC2 environment. The dataset comprised 11,000 frames, divided into 10-second gameplay chunks.
Efficiency and Accuracy
The Hebbian-trained encoder achieved lower prediction loss (measured by Chamfer loss) than self-supervised baselines. Compared to a standard frame-based ConvLSTM, the point-set model demonstrated extreme efficiency:
- 110x fewer activations
- 793x fewer FLOPs (Floating Point Operations)
Data Efficiency
The Hebbian approach proved much more robust when training data was limited. Even with only 10% of the training data, the model maintained its performance, whereas the error rate of standard self-supervised models spiked significantly.
Conclusion and Future Work
This research demonstrates that unsupervised Hebbian learning on point sets provides a computationally efficient and data-robust alternative to image-based models in complex RTS environments. Future work includes extending the model to handle dynamic unit counts (birth/death processes) and integrating these learned features into reinforcement learning agents to improve actual game-playing performance.
Summary updated