Classical decision theory operates under an elegant but physically impossible assumption: that thinking is free. In standard models of perfect rationality, an agent instantly calculates the expected utility of every possible action and selects the absolute optimum.
In reality, computation is a physical process. It takes time, consumes energy, and requires hardware cycles. If an autonomous drone spends ten seconds computing the mathematically perfect trajectory to avoid an oncoming obstacle, it will crash long before it finishes the calculation. As Eric Horvitz notes in Reasoning About Beliefs and Actions Under Computational Resource Constraints, real-world reasoning is often NP-hard and subject to strict deadlines; waiting for a complete analysis can render the resulting decision entirely useless.
To build truly intelligent systems, researchers have shifted from perfect rationality to frameworks that treat computational resources as finite assets that must be managed strategically.
Metareasoning: The Decision Theory of Thought
How can an agent decide how to think? The foundational answer lies in rational metareasoning: applying decision theory to the computational process itself.
Instead of treating search or inference as a fixed, unyielding algorithm, Russell et al. proposed in On Optimal Game-Tree Search using Rational Meta-Reasoning that internal deliberation steps should be evaluated based on their expected value. Every computational step has a cost (primarily the passage of time) and a potential benefit (improving the quality of the ultimate action chosen).
Formally, an agent must navigate a fundamental trade-off between the cost of internal deliberation and the expected improvement in decision quality, a mechanism explored deeply by Callaway et al. in Learning to select computations. Horvitz et al. termed this balance "Type II rationality" in Reflection and Action Under Scarce Resources: Theoretical Principles and Empirical Study, providing a framework for determining exactly when an agent should stop reflecting and start acting.
If the expected gain from expanding one more node in a search tree or running one more inference step is less than the cost of the time consumed, the metareasoner halts the computation and executes the best action found so far. Vidal et al. demonstrate in Agent Modeling Methods Using Limited Rationality that this trade-off is equally vital in multi-agent settings, where agents must balance the temporal cost of modeling opponents against the marginal strategic benefit it yields.
Bounded Optimality and the Emergence of Heuristics
If perfect rationality is unachievable, what should engineers optimize for? Russell et al. introduced the concept of bounded optimality in Provably Bounded Optimal Agents. A bounded optimal agent is not one that makes the globally perfect choice, but one whose program is provably the most effective possible given its specific hardware constraints and environment.
This perspective reframes how we view cognitive limitations and seemingly "irrational" biases. Halpern et al. demonstrate in Decision Theory with Resource-Bounded Agents that when you explicitly charge an agent for the complexity of its thoughts (modeling them as finite automata or resource-bounded Turing machines), behaviors like first-impression bias emerge as mathematically optimal shortcuts. Similarly, Halpern et al. show in Bounded Rationality in Las Vegas: Probabilistic Finite Automata Play Multi-Armed Bandits that simple, memory-limited strategies like "aspiration-level" heuristics perform near-optimally while drastically reducing cognitive overhead.
This suggests that human cognitive heuristics are not design flaws, but elegant solutions to resource constraints. Supporting this, Binz et al. showed in Where Do Heuristics Come From? that when recurrent neural networks are trained under strict resource constraints, they naturally discover a spectrum of heuristic strategies that mirror human behavioral data. Ortega et al. further validated this in Human Decision-Making under Limited Time, proving that under severe time pressure, human decision-making systematically falls back on learned prior patterns to minimize information-processing costs.
Mechanisms of Resource Conservation: Abstraction and Amortization
To operate effectively under what PEI WANG calls the "Assumption of Insufficient Knowledge and Resources" (AIKR) in Intelligence: From Definition to Design, agents must employ structural mechanisms to compress their problem spaces.
One powerful mechanism is state abstraction. Larsson et al. demonstrate in Hierarchical state abstractions for decision-making problems with computational constraints that grouping fine-grained environmental states into coarse, hierarchical abstractions emerges naturally when information-processing costs are minimized. By planning over a simplified map, the agent drastically reduces the computational energy required to find a viable path.
Another mechanism is amortization. Instead of building a complex mental model from scratch for every new problem, Nagy et al. propose in Analogy making as amortised model construction that agents use analogies to draw from a library of reusable, abstract modules. This spreads the high computational cost of model construction across many different experiences over an agent's lifetime.
In modern large language model (LLM) agents, this resource management manifests as a choice between action representations. Xu et al. identify a "cognitive bandwidth bottleneck" in The Cognitive Bandwidth Bottleneck: Shifting Long-Horizon Agent from Planning with Actions to Planning with Schemas. They demonstrate that as action spaces grow, listing and parsing every discrete action overwhelms the agent's fixed cognitive capacity. The optimal strategy shifts from planning with raw actions to planning with abstract "schemas" (templates with parameters), which dramatically lowers the processing overhead.
Engineering Bounded Systems
Translating these theoretical principles into physical systems requires explicit architectural constraints. Early work by Kurt Konolige in A Computational Theory of Belief Introspection enforced resource bounds directly on the number of derivation steps an inference engine could take, returning an "undecided" status rather than hanging indefinitely.
In modern robotics and cyber-physical systems, this is achieved by embedding resource awareness directly into execution loops. Traldi et al. show in Real-Time BDI Agents: A Model and Its Implementation how integrating real-time CPU scheduling and temporal planning directly into the Belief-Desire-Intention (BDI) cycle prevents agents from generating plans that exceed their parallel execution capacity. When crises occur, chasing an absolute optimum becomes impossible; as Sarathy et al. argue in Managing Trustworthiness in Advanced Autonomous Systems, trustworthy architectures must prioritize safe degradation over optimality, ensuring the agent remains responsive within its physical and temporal boundaries.
Ultimately, frameworks like the Boundedly Rational Inductive Agents (BRIAs) developed by Oesterheld et al. in A Theory of Bounded Inductive Rationality prove that agents operating within strict time-complexity bounds can still satisfy normative rationality criteria in the limit. By treating computation as a scarce, measurable resource, the field bridges the gap between idealized mathematical intelligence and the physical realities of silicon and survival.
Key Foundational Work
On Optimal Game-Tree Search using Rational Meta-Reasoning (1989) — contributed 1 concept elements to this direction.
Reflection and Action Under Scarce Resources: Theoretical Principles and Empirical Study (1989) — contributed 1 concept elements to this direction.
A Computational Theory of Belief Introspection (1985) — contributed 1 concept elements to this direction.