Reflection and Action Under Scarce Resources: Theoretical Principles and Empirical Study

Eric Horvitz, Gregory F. Cooper, David Heckerman

Abstract

We define and exercise the expected value of computation as a fundamental component of reflection about alternative inference strategies. We present a portion of Protos research focused on the interlacing of reflection and action under scarce resources, and discuss how the techniques have been applied in a high-stakes medical domain. The work centers on endowing a computational agent with the ability to harness incomplete characterizations of problem-solving performance to control the amount of effort applied to a problem or subproblem, before taking action in the world or turning to another problem. We explore the use of the techniques in controlling decision-theoretic inference itself, and pose the approach as a model of rationality under scarce resources.

Introduction

A fundamental challenge in artificial intelligence is the management of computational resources in real-time environments. Traditional "Type I" rationality focuses on finding the optimal action given a set of beliefs, often ignoring the cost of the time taken to reach those beliefs. However, in high-stakes domains such as intensive care medicine, the delay caused by complex probabilistic inference can be fatal. This paper introduces a framework for "Type II" rationality, where an agent accounts for its own finite resources—time and memory—when deciding whether to continue reasoning or to act immediately.

The Expected Value of Computation (EVC)

To formalize the trade-off between the benefits of additional computation and the costs of delay, we introduce the Comprehensive Value ($U_c$). This utility function integrates the object-level utility of an action in the world with the inference-related costs, such as the disutility of time elapsed.

The core metric for metareasoning is the Expected Value of Computation (EVC). EVC represents the expected net change in comprehensive value resulting from a specific allocation of computational resources ($\Delta t$). If the EVC is positive, the expected improvement in the decision's quality outweighs the cost of the delay.

Metareasoning Control Loops

The agent employs a metareasoning control loop to monitor its own progress. By "probing" the EVC at discrete intervals, the agent can decide dynamically when to terminate computation.

The metareasoning control loop for evaluating the Expected Value of Computation (EVC).

As shown in the control loop above, the agent evaluates whether the expected gain from the next chunk of computation is positive. If so, it continues to refine its beliefs; otherwise, it executes the action currently deemed optimal. This approach treats "computation" as an action itself, subject to the same decision-theoretic scrutiny as physical interventions.

Empirical Evaluation: The ALARM Network

The framework was tested using the ALARM (A Logical Alarm Reduction Mechanism) network, a complex belief network representing 37 variables in ICU physiology. This domain provides a rigorous testbed for decision-making under pressure, specifically regarding life-critical actions like mechanical ventilation.

The ALARM network (top) and the decision model for mechanical ventilation (bottom).

The study utilized bounded conditioning, an approximation method that provides narrowing upper and lower bounds on probabilities. As computation progresses, these bounds tighten. A key insight is the $p^*$ threshold: the probability at which the optimal action changes. If the current bounds for a critical variable (e.g., respiratory status) do not straddle $p^*$, the optimal action is already determined, and further computation provides zero utility.

Results and Rationality

Empirical results on the ALARM network revealed that the uncertainty interval typically follows an exponential decay model. In the experimental setup, the "half-life" of uncertainty was approximately 36 seconds. This predictability allows the agent to estimate the future value of computation accurately.

By formalizing the "cost of thinking," this research moves beyond heuristic resource management toward a rigorous mathematical model of bounded rationality. The agent can often reach a decision threshold and act correctly long before a full, exact inference is completed, demonstrating the practical necessity of Type II rationality in complex, real-world environments.

Conclusion

The interlacing of reflection and action through EVC probes provides a robust mechanism for agents operating under scarce resources. By treating computation as a costly resource, AI systems can achieve a higher level of practical utility in time-sensitive domains, ensuring that "perfect" reasoning does not become the enemy of "timely" action.

Summary updated

Abstract

We define and exercise the expected value of computation as a fundamental component of reflection about alternative inference strategies. We present a portion of Protos research focused on the interlacing of reflection and action under scarce resources, and discuss how the techniques have been applied in a high-stakes medical domain. The work centers on endowing a computational agent with the ability to harness incomplete characterizations of problem-solving performance to control the amount of effort applied to a problem or subproblem, before taking action in the world or turning to another problem. We explore the use of the techniques in controlling decision-theoretic inference itself, and pose the approach as a model of rationality under scarce resources.
BibTeX
@inproceedings{horvitz1989reflection,
  author = {Eric Horvitz and Gregory F. Cooper and David Heckerman},
  title = {{Reflection and Action Under Scarce Resources: Theoretical Principles and Empirical Study}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1989},
}
Figure