HLSMAC: A New StarCraft Multi-Agent Challenge for High-Level Strategic Decision-Making

Xingxing Hong, Yungong Wang, Dexin Jin, Ye Yuan, Ximing Huang, Zijian Wu, Wenxin LiarXiv 2025 · arXiv · OpenReview

TL;DR

This paper introduces HLSMAC, a novel benchmark for StarCraft II that shifts the focus from micromanagement to high-level strategic decision-making by incorporating classical Chinese "Thirty-Six Stratagems." It also proposes new evaluation metrics beyond win rate to better assess strategic intelligence, demonstrating that current MARL and LLM agents struggle with these complex scenarios.

What Problem Does This Paper Address?

Multi-agent reinforcement learning (MARL) has seen significant progress, largely thanks to complex environments like StarCraft II. However, existing benchmarks, particularly SMAC (StarCraft Multi-Agent Challenge), primarily focus on low-level micromanagement (e.g., unit control, targeting). This leaves a critical gap in evaluating an agent's ability to perform high-level strategic planning, coordination, and deception – skills crucial for complex real-world tasks. Furthermore, current benchmarks often fail to leverage established human strategic wisdom, relying instead on emergent learning from environmental interactions.

Methods

The core contribution is the HLSMAC benchmark, which translates 12 of the "Thirty-Six Stratagems" into distinct StarCraft II scenarios. These scenarios are designed to challenge agents with:

  • Larger Maps and Richer Terrain: Enabling longer movement, more route options, and diverse terrain features to encourage strategic positioning.
  • Expanded Unit and Structure Abilities: Moving beyond basic move/stop/attack to include abilities like Burrow, Hallucination, and Nydus Worm transport, which are crucial for deception and tactical maneuvering.
  • Diverse Opponent Policies: Implementing triggers for more realistic and varied enemy behaviors.
  • Redefined Game Termination Conditions: Shifting from simple unit elimination to more nuanced objectives that require strategic planning.

To evaluate agents on HLSMAC, the authors propose novel metrics beyond the standard win rate:

  • Critical Target Advancement (CTA): Measures how effectively agents move towards key objectives. This includes:
  • Target Proximity Frequency (TPF): How often units are near critical targets.
  • Target Directional Alignment (TDA): How directly units move towards targets.
  • Ability Utilization Frequency (AUF): Tracks how often agents use special abilities.
  • Critical Target Damage (CTD): Measures damage dealt to critical enemy targets.
  • Unit Survival Rate (USR): Assesses an agent's ability to preserve its units.

The benchmark is designed to be compatible with popular MARL frameworks like PyMARL and LLM frameworks like LLM-PySC2.

Datasets

The primary "dataset" is the HLSMAC benchmark itself, consisting of 12 custom StarCraft II scenarios. These scenarios are built using official StarCraft II game data and maps, adapted to incorporate the principles of the Thirty-Six Stratagems. The authors also evaluate against existing state-of-the-art MARL algorithms and LLM-based agents.

Key Insights and Ideas

  • Human Strategic Wisdom is Underexplored: Classical stratagems offer a rich source of high-level strategic principles that can be directly translated into AI evaluation scenarios, moving beyond purely emergent behavior.
  • Micromanagement vs. Strategy: Existing benchmarks are heavily skewed towards micromanagement. HLSMAC explicitly aims to test strategic planning, deception, and coordination, revealing that current agents often fail when these are prioritized.
  • Win Rate is Insufficient: Traditional win rates can be misleading. An agent might win by exploiting a specific weakness or through brute force, without truly understanding the underlying strategy. New metrics are crucial for a deeper evaluation.
  • LLMs Struggle with Strategic Nuance: Even advanced LLM agents (like GPT-3.5) show limited tactical awareness and often fail to generate effective action sequences to accomplish complex strategic goals in HLSMAC.

Key Results

  • High Failure Rate for MARL Agents: Approximately 80% of algorithm-scenario combinations resulted in zero win rates, highlighting the difficulty of HLSMAC.
  • LLM Agents Also Struggle: GPT-3.5 agents performed poorly, failing to comprehend most scenarios and exhibiting ineffective action selection.
  • New Metrics Show Promise: The proposed metrics (TPF, TDA, CTD, AUF, USR) demonstrate strong explanatory power (R² ≥ 0.6) for predicting performance in many scenarios, offering a more nuanced view than win rate alone. For instance, CTD showed the highest correlation with win rate.
  • Distinguishing Human vs. AI Behavior: AUF, while having a weaker correlation with win rate, proved valuable in distinguishing purposeful human ability usage from the often aimless or repetitive use by current AI methods.

Limitations and Future Work

  • Limited LLM Exploration: The current LLM evaluation focused on interpreting stratagem names and brief explanations. Future work could explore more detailed prompts and LLM capabilities.
  • Need for Automated Scenario Generation: Creating these complex scenarios is time-consuming. Developing methodologies for automated scenario generation would be beneficial.
  • Specialized Algorithms: The paper suggests the need for developing specialized algorithms tailored to HLSMAC's unique challenges, particularly those requiring high-level strategic reasoning.
  • Further Exploration of Metrics: While promising, further research into the interpretation and application of the new metrics is warranted.

Summary updated

Abstract

Benchmarks are crucial for assessing multi-agent reinforcement learning (MARL) algorithms. While StarCraft II-related environments have driven significant advances in MARL, existing benchmarks like SMAC focus primarily on micromanagement, limiting comprehensive evaluation of high-level strategic intelligence. To address this, we introduce HLSMAC, a new cooperative MARL benchmark with 12 carefully designed StarCraft II scenarios based on classical stratagems from the Thirty-Six Stratagems. Each scenario corresponds to a specific stratagem and is designed to challenge agents with diverse strategic elements, including tactical maneuvering, timing coordination, and deception, thereby opening up avenues for evaluating high-level strategic decision-making capabilities. We also propose novel metrics across multiple dimensions beyond conventional win rate, such as ability utilization and advancement efficiency, to assess agents' overall performance within the HLSMAC environment. We integrate state-of-the-art MARL algorithms and LLM-based agents with our benchmark and conduct comprehensive experiments. The results demonstrate that HLSMAC serves as a robust testbed for advancing multi-agent strategic decision-making.
BibTeX
@misc{hong2025hlsmac,
  author = {Xingxing Hong and Yungong Wang and Dexin Jin and Ye Yuan and Ximing Huang and Zijian Wu and Wenxin Li},
  title = {{HLSMAC: A New StarCraft Multi-Agent Challenge for High-Level Strategic Decision-Making}},
  year = {2025},
  eprint = {2509.12927},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.12927},
}

Loading related content...

Figure