Abstract: Game AI is of great importance as games are simulations of reality. Recent research on game AI has shown much progress in various kinds of games, such as console games, board games and MOBA games. However, the exploration in RTS games remains a challenge for their huge state space, imperfect information, sparse rewards and various strategies. Besides, the typical card-based RTS games have complex card features and are still lacking solutions. We present a deep model SEAT (selection-attention) to play card-based RTS games. The SEAT model includes two parts, a selection part for card choice and an attention part for card usage, and it learns from scratch via deep reinforcement learning. Empirical results show that the SEAT model agent makes it to reach a high winning rate against rule-based agents and decision-tree-based agent.
Introduction
Real-Time Strategy (RTS) games represent a significant frontier for Artificial Intelligence. Unlike board games with discrete states, RTS games like Clash Royale present massive state spaces, imperfect information, and the need for high-precision spatial decision-making. Card-based RTS games add further complexity through unique card attributes (HP, damage, abilities) and the critical management of resources like "elixir."
Existing heuristic methods often fail to capture the nuances of defensive "pulling" or aggressive "bridge spam" tactics. To address this, researchers have developed SEAT (Selection-Attention), a Deep Reinforcement Learning (DRL) framework that masters these complexities by decoupling the decision process.
The SEAT Model Architecture
The core innovation of the SEAT model is its "divide and conquer" approach to the action space. Instead of learning a single mapping from state to every possible card-coordinate pair, the model splits the task into two specialized components:
- The Selection Part: This component determines which card to play. It processes the battlefield state—represented as a multi-channel grid of unit health, class, and faction—alongside specific card features using a combination of Convolutional Neural Networks (CNN) and Multi-Layer Perceptrons (MLP).
- The Attention Part: Once a card is chosen, this component determines where to place it. By generating an "attention map" over the battlefield, the model identifies optimal deployment coordinates based on enemy concentrations and tower health.
Experimental Setup and Training
The model was trained from scratch in a custom-built Clash Royale simulation. A key training strategy involved the "abandoning-rate" ($\epsilon$). By encouraging the model to "do nothing" 50% of the time during early training, the agent learned the value of patience and elixir conservation—a critical skill for high-level play.
The evaluation utilized a standardized deck of eight cards, including multi-unit troops (Ninja, BatSwarm), tower-targeted units (ApeWarior), and area-of-effect spells (FireBall, Zap). Performance was measured against three benchmark agents: Aggressive Rule-Based (ARB), Defensive Rule-Based (DRB), and a sophisticated Decision-Tree (DT) agent.
Results and Tactical Emergence
The SEAT-agent demonstrated overwhelming superiority over traditional AI approaches. It achieved a 90% win rate against both aggressive and defensive rule-based agents. Even against the more complex Decision-Tree agent, which combines multiple strategies, SEAT maintained a robust 70% win rate.
Beyond raw statistics, the model exhibited emergent tactical behavior. It learned to use low-cost units to distract high-damage enemies and timed its spells to maximize area-of-effect damage, effectively mimicking "pro" human maneuvers without explicit instruction.
Conclusion
The SEAT model provides a scalable solution for the complex action spaces of card-based RTS games. While currently limited to fixed decks and simulated opponents, the success of the selection-attention decoupling suggests a path forward for more generalized game AI. Future work will focus on expanding the model's capability to handle dynamic deck building and testing against high-level human players.
Summary updated