Introduction
Multiplayer Online Battle Arena (MOBA) games like Honour of Kings present a significant challenge for artificial intelligence due to their massive state spaces, imperfect information ("fog of war"), and sparse, delayed rewards. While previous breakthroughs in game AI have excelled at micro-level execution—the mechanics of fighting and dodging—they often struggle with macro strategy, which involves high-level planning and team-wide coordination. This paper introduces the Hierarchical Macro Strategy (HMS) model, a supervised learning framework designed to provide AI agents with the "big picture" planning skills necessary to compete at an elite level.
The HMS Architecture
The authors propose a two-layer hierarchical model integrated into a multi-task neural network. This architecture allows the agent to process global game information and hero attributes to make strategic decisions.
- Phase Layer: This acts as an auxiliary task that recognizes the current stage of the game (e.g., opening, laning, or late-game). It identifies these phases by predicting which major resources, such as turrets or dragons, are currently under threat.
- Attention Layer: This is the core strategic component. It frames macro strategy as a spatial attention problem, predicting the optimal region on a 12x12 map grid for an agent to occupy.
- Imitated Cross-Agent Communication: To solve the coordination problem, the model uses a novel mechanism where agents share their predicted destinations. During training, an agent receives the "attention labels" of its allies, allowing it to learn how to coordinate movements implicitly without a complex, handcrafted protocol.
Training and Methodology
The HMS model was trained using a massive dataset from Honour of Kings, consisting of approximately 300,000 replays from professional leagues and high-level matches. This resulted in 250 million individual training instances. The model extracts 85 visual features and 181 attribute features (such as gold, level, and hero IDs) to inform its decisions. By employing supervised learning from professional replays, the researchers captured complex human macro-intuitions that are difficult to define via traditional mathematical reward functions used in reinforcement learning.
Key Results and Insights
The HMS-powered AI was evaluated against elite human teams and in ablation studies, yielding several key insights:
- Elite Performance: The 5-AI team achieved a 48% win rate against human players ranked in the top 1% (King rank) over 250 games.
- Macro vs. Micro: In head-to-head matches, the HMS model defeated a version of the AI without macro strategy 75% of the time. The macro-less AI tended to focus only on nearby targets and failed to push lanes effectively.
- The Power of Communication: The version with "Imitated Communication" won 62.5% of matches against a version without it. Experts noted that high-quality opening strategies increased from 22% to 83% with communication enabled.
- Phase Impact: Removing the Phase Layer dropped the win rate to 35% against the full HMS model, as the AI lost its sense of timing for critical objectives like the "Baron" boss.
Conclusion
The Hierarchical Macro Strategy model demonstrates that explicit modeling of game phases and spatial attention is crucial for mastering complex multi-agent environments. By learning from human experts, the HMS framework enables AI to coordinate effectively and make high-level strategic decisions. Future work may involve combining this supervised approach with reinforcement learning and Monte Carlo Tree Search (MCTS) to further refine the AI's long-term planning and evaluation capabilities.
Summary updated