Abstract
This paper introduces LLM-Nash Games, a novel game-theoretic framework where strategic interaction occurs at the level of reasoning prompts rather than final actions. By modeling how Large Language Models (LLMs) deliberate before deciding, the research captures "bounded rationality" in AI agents. The study demonstrates that optimal reasoning within the constraints of a language model does not always align with classical Nash equilibria, revealing a fundamental gap between theoretical perfect play and emergent AI behavior.
Introduction
Traditional game theory is built upon the "Rational Actor" model, which assumes agents possess perfect information, infinite computational power, and a singular goal of utility maximization. However, Large Language Models (LLMs) do not function as pure optimizers. Instead, they generate responses based on context, internal biases, and the specific structure of their instructions. This creates a significant discrepancy between theoretical predictions and actual AI performance.
This research addresses the need for a mathematical framework that treats the reasoning process—the prompt—as the primary strategic choice. By shifting the focus from actions to "mindsets," the author provides a way to study how cognitive constraints and linguistic framing affect the outcomes of strategic games.
The LLM-Nash Framework
The paper proposes a two-tier equilibrium structure to model agent interaction:
- The Mindset: An agent's strategic state is defined as a tuple $(I, x, heta)$, where $I$ is the information available to the agent, $x$ is the natural language prompt (the reasoning strategy), and $ heta$ represents the internal architecture and state of the LLM.
- Reasoning Equilibrium: Rather than selecting a discrete action $a$, players choose a prompt $x$ from a defined "prompt space." An equilibrium is reached when no player can improve their expected payoff by unilaterally changing their reasoning prompt.
- Behavioral Output: Actions are viewed as emergent behaviors triggered by the chosen reasoning prompt.
To evaluate the richness of these interactions, the paper introduces Strategic Expressiveness, a metric that ranks how flexible an agent's mindset is based on the variety of behaviors it can produce given its internal constraints.
Key Theoretical Insights
The framework reveals several critical insights into the nature of AI strategic behavior:
- The Utility Gap: The paper proves that reasoning-level decision-making is almost always "coarser" than behavioral-level decision-making. Because an LLM is limited by its training and the prompts available to it, it can rarely achieve the same theoretical maximum utility as a "perfect" rational agent.
- Mindsets as Constraints: Bounded rationality is explicitly modeled as the limits of the prompt space and the LLM’s internal "worldview" ($\theta$). This makes the concept of limited intelligence mathematically tractable.
- The Complexity Paradox: Interestingly, the author finds that a more "expressive" mindset (more complex reasoning) does not guarantee a better outcome. In competitive environments, complex reasoning can lead to "overfitting" to an opponent's perceived strategy, potentially making the agent more exploitable.
Case Study: Rock-Paper-Scissors
To validate the framework, the author utilized a simulated case study of Rock-Paper-Scissors (RPS). Agents were given empirical frequencies of an opponent's past moves and had to choose between different reasoning prompts, such as "Exploit the opponent's bias" or "Choose a move uniformly."
Results
- Divergence from Nash: In classical game theory, the Nash equilibrium for RPS is a perfectly uniform 1/3 distribution for each move. However, when agents were restricted to specific reasoning prompts, they reached a "Reasoning Equilibrium" that deviated from this norm, often settling on biased distributions (e.g., playing Paper 60% of the time).
- Stable Biases: These biases remained stable because the available prompt space did not allow the agents to "reason" their way back to a perfectly uniform strategy.
- Epistemic Learning: The results suggest that for LLMs to reach classical optimality, they must undergo "epistemic learning"—evolving their "cognitive scaffolding" or learning new ways to think, rather than just updating their move frequencies.
Limitations and Future Work
The current framework assumes the underlying LLM ($\theta$) is fixed. Future research could explore how fine-tuning or switching between different models changes the resulting equilibrium. Additionally, while the theory relies on a defined "prompt space," the near-infinite nature of natural language makes mapping every possible reasoning strategy difficult in practice.
The author suggests extending this work to "correlated reasoning," where agents sharing the same foundation model might exhibit naturally aligned cognitive structures. Another promising direction is the integration of neurosymbolic learning to help agents autonomously discover higher-order reasoning prompts during gameplay.
Summary updated