Unsupervised Hebbian Learning on Point Sets in StarCraft II

Beomseok Kang, Harshit Kumar, Saurabh Dash, Saibal Mukhopadhyay

Abstract

Artificial Intelligence in Real-Time Strategy (RTS) games like StarCraft II is hindered by the computational cost of processing sparse game maps as dense images. This paper introduces a biologically inspired, unsupervised Hebbian learning method that treats game units as sparse point sets. By employing a novel "neuron activity aware" learning rule and k-Winner-Takes-All (k-WTA) competition, the model achieves superior prediction accuracy for unit movements. Notably, the approach requires 110x fewer activations and 793x fewer FLOPs than traditional frame-based models, while demonstrating significantly better data efficiency.

Introduction

StarCraft II maps are inherently sparse. In a typical 256x256 resolution map, even a maximum army occupies only about 0.6% of the total pixels. Processing these vast empty areas using standard Convolutional Neural Networks (CNNs) is computationally wasteful.

The sparsity of StarCraft II maps: (a) shows the full game map, while (b) highlights how few pixels are actually occupied by units.

The authors propose that representing units as a point set—a collection of (x, y) coordinates—is a far more efficient way to model game dynamics. However, learning meaningful representations from such sets without explicit supervision remains a challenge.

Methodology

The researchers developed an Encoder-LSTM-Decoder architecture. The core innovation lies in the unsupervised training of the encoder, which utilizes principles from biological neural networks.

Hebbian Learning on Point Sets

Instead of standard backpropagation, the encoder uses a variant of Grossberg’s "instar rule." In this unsupervised method, weights are updated based on the proximity of input points to weight vectors. To ensure sparse and distinct representations, a k-Winner-Takes-All (k-WTA) mechanism is used, where only the 'k' most active neurons fire for a given input.

Neuron Activity Aware Learning

Hebbian learning often suffers from "dead" neurons or over-active neurons that dominate the representation. To counter this, the authors derived an optimal neuron activity level ($p^* = k/d$) using Lagrange multipliers. They introduced anti-Hebbian learning to push the weights of over-active neurons away from data points, while standard Hebbian learning pulls under-active neurons toward them.

The encoder architecture transforms a variable number of unit positions into a fixed-dimension latent vector.

The resulting latent vectors are fed into an LSTM to model temporal changes and a Decoder to reconstruct the predicted future positions of units.

Experimental Results

The model was trained on StarCraft II replay data (Terran vs. Terran) extracted via the PySC2 environment. The dataset comprised 11,000 frames, divided into 10-second gameplay chunks.

Efficiency and Accuracy

The Hebbian-trained encoder achieved lower prediction loss (measured by Chamfer loss) than self-supervised baselines. Compared to a standard frame-based ConvLSTM, the point-set model demonstrated extreme efficiency:

  • 110x fewer activations
  • 793x fewer FLOPs (Floating Point Operations)

Data Efficiency

The Hebbian approach proved much more robust when training data was limited. Even with only 10% of the training data, the model maintained its performance, whereas the error rate of standard self-supervised models spiked significantly.

Conclusion and Future Work

This research demonstrates that unsupervised Hebbian learning on point sets provides a computationally efficient and data-robust alternative to image-based models in complex RTS environments. Future work includes extending the model to handle dynamic unit counts (birth/death processes) and integrating these learned features into reinforcement learning agents to improve actual game-playing performance.

Summary updated

Abstract

Learning the evolution of real-time strategy (RTS) game is a challenging problem in artificial intelligent (AI) system. In this paper, we present a novel Hebbian learning method to extract the global feature of point sets in StarCraft II game units, and its application to predict the movement of the points. Our model includes encoder, LSTM, and decoder, and we train the encoder with the unsupervised learning method. We introduce the concept of neuron activity aware learning combined with k-Winner-Takes-All. The optimal value of neuron activity is mathematically derived, and experiments support the effectiveness of the concept over the downstream task. Our Hebbian learning rule benefits the prediction with lower loss compared to self-supervised learning. Also, our model significantly saves the computational cost such as activations and FLOPs compared to a frame-based approach.
BibTeX
@inproceedings{kang2022unsupervised,
  author = {Beomseok Kang and Harshit Kumar and Saurabh Dash and Saibal Mukhopadhyay},
  title = {{Unsupervised Hebbian Learning on Point Sets in StarCraft II}},
  booktitle = {IJCNN},
  year = {2022},
  doi = {10.1109/ijcnn55064.2022.9892259},
}
Figure