Abstract
This paper establishes a formal link between information-theoretic bounded rationality and game theory. By applying the Legendre-Fenchel transform to the free energy objective, the authors demonstrate that a single agent's boundedly rational behavior is mathematically equivalent to playing a minimax game against a hypothetical adversary. This perspective provides a strategic justification for stochastic policies, framing them as optimal defenses against potential environmental fluctuations or adversarial interference.
Introduction
In classical decision theory, "perfectly rational" agents are assumed to possess the infinite computational resources required to identify and execute the single best action. However, real-world agents—whether biological or artificial—operate under inherent constraints. Bounded rationality models these limitations by incorporating costs for information processing or policy complexity.
A prominent framework in this field is Information-Theoretic (IT) Bounded Rationality, where agents maximize "free energy." This objective balances the pursuit of utility with a penalty for deviating from a prior policy, typically measured via KL-divergence. While mathematically elegant, the strategic significance of this information cost has remained elusive. This research reveals that such costs are not merely computational "friction" but represent a robust response to a potentially hostile environment.
The Free Energy Objective
The core of IT bounded rationality is the maximization of the free energy functional. This involves taking the expected utility of a policy and subtracting a cost proportional to its complexity (the KL-divergence from a reference prior).
As shown in the figure above, the free energy objective can be visualized on a probability simplex. The parameter $\beta$ acts as a trade-off: as $\beta \to \infty$, the agent becomes perfectly rational (maximizing expected utility); as $\beta \to 0$, the agent defaults to its prior. In between, the agent exhibits "softmax" behavior, distributing probability across actions based on their relative values.
Methodology: The Adversarial Dual
To uncover the strategic meaning of this objective, the researchers employ the Legendre-Fenchel transform. This mathematical tool allows for the representation of a convex function in terms of its dual. By applying this transform to the KL-divergence term in the free energy objective, the authors decompose the single-agent optimization problem into a two-player minimax game:
- The Agent: Seeks to maximize its net utility.
- The Imaginary Adversary: Seeks to minimize the agent's utility by imposing costs on specific actions, while incurring a penalty for the magnitude of these costs.
This transformation proves that the "information cost" in bounded rationality is functionally identical to the presence of an adversary who can manipulate payoffs.
Key Insights
Stochasticity as a Shield
In traditional AI, stochastic behavior is often dismissed as noise or uncertainty. This paper suggests a more profound interpretation: for a bounded agent, acting randomly is a defensive maneuver. By spreading probability across multiple actions, the agent prevents an adversary from "zeroing in" on and sabotaging a single deterministic choice. Stochasticity provides robustness.
The Indifference Principle
A striking result of this work is the Indifference Principle (Theorem 4). At the optimal equilibrium, the adversary chooses costs such that the agent becomes indifferent to the choices it makes with non-zero probability. This is a hallmark of a Nash Equilibrium in game theory. It implies that bounded rationality represents a state of equilibrium with a hypothetical hostile environment, rather than just a "lazy" approximation of perfect rationality.
The figure above illustrates how different policies $p(x)$ react to utility $U(x)$ and the resulting adversarial costs $C(x)$. The final policy balances the raw utility against the potential for an adversary to reduce that utility.
Conclusion
By bridging information theory and game theory, this paper provides a robust foundation for understanding boundedly rational behavior. The adversarial interpretation suggests that information-theoretic constraints are a natural way to model risk-sensitivity and robustness. Future work may extend this framework to multi-step reinforcement learning tasks and explore its applications in actual adversarial machine learning contexts.
Summary updated