Abstract
As global militaries explore Large Language Models (LLMs) for decision support in high-stakes environments, the reliability and consistency of these models remain under-scrutinized. This paper investigates the semantic inconsistency of LLMs in free-form military crisis simulations. We introduce a quantitative metric based on BERTScore to measure divergence in open-ended recommendations. Our findings indicate that state-of-the-art models—including GPT-4 and Claude 3.5 Sonnet—exhibit significant semantic inconsistency when presented with identical or semantically equivalent scenarios. Notably, prompt sensitivity often induces greater inconsistency than internal sampling temperature. These results suggest that current LLMs lack the stability required for autonomous or advisory roles in military command.
Introduction
The integration of artificial intelligence into military "wargaming" has traditionally relied on constrained environments where models select from pre-defined actions. However, real-world application demands the flexibility of natural language, where commanders query models for nuanced analysis and strategic recommendations. This shift to "free-form" decision-making introduces a significant challenge: how to measure if a model is being consistent when its responses are linguistically diverse.
Two responses may use different terminology to suggest the same strategy, or conversely, use similar language to recommend diametrically opposed actions (e.g., "de-escalate" vs. "deploy force"). This study addresses this gap by developing a robust framework to quantify semantic inconsistency in open-ended military simulations.
Quantifying Inconsistency via BERTScore
To measure inconsistency without the limitations of multiple-choice formats, we utilize a metric derived from BERTScore. BERTScore leverages contextual embeddings to compare the semantic similarity of two text sequences. We define the Inconsistency Score as $1 - \text{BERTScore}$.
To ensure this metric captures genuine shifts in meaning rather than mere stylistic variation, we validated it using four types of "ablations" on the TruthfulQA dataset:
- Lexical Substitution: Replacing words with synonyms.
- Syntactic Restructuring: Altering sentence structure.
- Addition of Irrelevance: Appending non-sequitur information.
- Semantic Shift: Modifying the core meaning of the response.
As illustrated in Figure 1, the metric effectively separates semantic shifts from meaning-preserving changes. Based on these results, an inconsistency score $\geq 0.25$ was established as the threshold for a genuine change in the recommended course of action.
Experimental Design
We evaluated five state-of-the-art models: Claude 3.5 Sonnet, GPT-3.5 Turbo, GPT-4, GPT-4o, and GPT-4o mini. The primary testbed was a realistic, fictitious crisis simulation involving a maritime dispute in the Taiwan Strait. Models were presented with an "Initial Setting" and two subsequent "Continuations" (a "Status Quo" path and a "Revisionist" path).
Beyond standard temperature sampling, we tested "Prompt Sensitivity" by creating multiple versions of the same scenario:
- Level One: 20 semantically identical prompts (paraphrased).
- Level Two: 20 semantically different but functionally similar prompts (e.g., different geographic regions with the same strategic stakes).
Results and Analysis
Our analysis reveals a universal trend of inconsistency across all tested models. Even at a deterministic temperature ($T=0$), models frequently provided divergent advice when the same scenario was phrased slightly differently.
As shown in Figure 4, GPT-4 emerged as the most inconsistent model in free-form responses, frequently recommending contradictory actions for the same situation. For instance, in one iteration, GPT-4 suggested diplomatic withdrawal, while in another, it authorized the use of non-lethal force and significant defense spending. While Claude 3.5 Sonnet and GPT-4o mini demonstrated higher relative stability, their scores still remained well above the semantic variation threshold.
Furthermore, we found that prompt sensitivity is a more significant driver of chaos than internal randomness. The divergence caused by semantically identical prompt variations often exceeded the divergence caused by increasing the model's temperature. This suggests that the "reasoning" path of the model is highly fragile and susceptible to minor linguistic cues.
Conclusion
The high-stakes nature of military decision-making leaves little room for the level of unpredictability observed in this study. The fact that anonymizing participants or using deterministic sampling does not resolve these inconsistencies indicates a fundamental instability in how LLMs process complex strategic scenarios. We recommend extreme caution and rigorous further testing before LLMs are integrated into any chain of command or high-stakes advisory role.
Summary updated