TL;DR
While defense organizations increasingly test large language models (LLMs) for strategic planning and wargaming, this paper demonstrates that LLMs provide wildly inconsistent free-form advice during simulated military crises. Even with zero temperature ($T=0$), trivial rewordings of prompts cause models to recommend conflicting courses of action—from aggressive military posturing to immediate retreats.
What Problem Does This Paper Address?
Governments and defense agencies are exploring LLMs to assist military commanders, support wargaming, and accelerate crisis decision-making under the assumption that AI can offer faster, objective analysis. However, prior evaluations predominantly constrained models to multiple-choice options (e.g., choosing among pre-defined escalation ladders).
In real-world deployment, humans interact with LLMs using unconstrained natural language. Evaluating free-form text has historically been difficult because measuring semantic differences without relying on pre-set choices is challenging. This paper tackles how to quantitatively evaluate free-form strategic decision-making inconsistency and investigates whether frontier LLMs are consistent enough to be safely trusted in high-stakes scenarios.
Methods
- BERTScore-Based Inconsistency Metric: The authors define inconsistency as $1 - \text{BERTScore}$ (using rescaled F1 scores from DeBERTa-xlarge fine-tuned on MNLI). They validate that this metric ignores surface-level syntactic reordering or synonym substitution while sensitively registering actual semantic shifts.
- Wargame Simulation Setup: Built upon an expert-designed geopolitical crisis simulation set in 2026 over the Taiwan Strait (between the US and China). Models act as presidential advisors recommending free-form courses of action.
- Evaluation Dimensions:
- Prompt Sensitivity at $T=0$: Evaluates greedy outputs under Level One ablations (semantically identical rewordings of the prompt) and Level Two ablations (changing roles, decision countries, or conflict locations).
- Temperature Sampling ($T > 0$): Measures variance across repeated stochastic samples from $T=0.2$ to $T=1.2$.
- Escalation Continuations: Tests whether escalating situations (a "status quo" vs. an aggressive "revisionist" scenario following a naval misfire) constrain inconsistency.
- Country Anonymization: Replaces national names with generic color identifiers to determine whether geopolitical biases drive variance.
- Model Suite: Evaluates GPT-3.5 Turbo, GPT-4, GPT-4o, GPT-4o mini, and Claude 3.5 Sonnet.
Datasets
- Taiwan Strait Crisis Benchmark: An expert-modeled military wargame scenario simulating high-tension naval encounters and rules-of-engagement friction.
- TruthfulQA: Used to validate the BERTScore inconsistency metric against controlled text ablations (lexical substitution, syntactic restructuring, irrelevance insertion, and semantic shift).
- Synthetic Alternative Conflict Scenarios: Four additional crisis vignettes created for Level Two ablations (US/Russia in Ukraine, US/Russia in the Arctic Circle, France/Pakistan over India, and Greece/Turkey in Cyprian waters).
- Mental Healthcare Safety Dataset: External clinical evaluation dataset (Grabb et al., 2024) containing suicide- and psychosis-related dialogues with human expert annotations ("safe," "borderline," "unsafe"), used to test cross-domain generalization.
Key Insights and Ideas
- Paraphrasing Trumps Temperature: For most models, simply changing prompt phrasing without changing its meaning at $T=0$ introduced inconsistency comparable to—or exceeding—sampling at stochastic temperatures ($T=0.2$ to $T=0.6$).
- Contradictory Operational Advice: Under identical crisis briefs, the same model at $T=1.0$ alternately recommended de-escalating by pulling the naval strike group back and authorizing non-lethal measures, or aggressively deploying overwhelming force and increasing aerial patrols.
- Inconsistency is Not Merely Country Bias: Anonymizing nation names with placeholder colors did not significantly reduce inconsistency, showing that volatility is rooted in strategic reasoning instability rather than localized country priors.
- Natural Language Entailment Fails on Multi-Sentence Text: The authors tried bi-directional entailment clustering (semantic entropy), but found it unviable for multi-sentence recommendations: real responses contain bundles of diverse actions where pairwise comparisons almost always yield "neutral" relations rather than direct logical entailment or contradiction.
Key Results
- Validation Thresholds: Controlled tests showed lexical substitutions scored an inconsistency of $0.08$ and syntactic restructuring scored $0.17$, while semantic shifts averaged $0.37$. The authors conservatively established scores $\ge 0.25$ as indicating meaningful semantic divergence.
- Level One Ablations ($T=0$): All tested models surpassed the $0.25$ divergence threshold under semantically identical prompts. GPT-3.5 Turbo showed the highest inconsistency ($\approx 0.52$), followed by GPT-4o ($\approx 0.48$), GPT-4 ($\approx 0.44$), and GPT-4o mini ($\approx 0.38$).
- Temperature Effects ($T > 0$): Inconsistency increased monotonically with temperature. Even at $T=0.2$, inconsistency scores exceeded $0.35$ for all models. At $T=1.0$, inconsistency ranged between $0.50$ and $0.72$, with GPT-4 exhibiting the highest variance.
- Crisis Escalation Impact: When moving from the open-ended initial crisis to specific incident continuations, inconsistency dropped (e.g., Claude 3.5 Sonnet fell from $\approx 0.58$ to $\approx 0.35$ in the revisionist branch), indicating that narrowing the decision space reduces output dispersion, though semantic instability remains non-trivial.
- Cross-Domain Validation: Applied to mental health chatbot dialogues, the inconsistency metric separated expert-labeled "safe" and "unsafe" responses with statistical significance.
Limitations and Future Work
- Domain Scope: The study primarily focuses on one primary crisis scenario (Taiwan Strait) involving three main actors; broader wargaming archetypes (e.g., ground wars, cyber-only conflicts) remain to be tested.
- Commercial vs. Specialized Models: Evaluations were limited to off-the-shelf foundation models; fine-tuned defense models or models augmented with doctrine-specific retrieval-augmented generation (RAG) may exhibit different behaviors.
- Ablation Confounders: Level Two prompt variations assumed constant decision-making difficulty across different world regions, which may be confounded by varying latent pre-training data density.
- Alternative Text Similarity Metrics: While BERTScore was verified qualitatively, exploring optimal-transport or LLM-as-a-judge evaluators (e.g., MoverScore, G-Eval) represents an avenue for future benchmark refinement.
Summary updated