Raising Expectations in GDA Agents Acting in Dynamic Environments

Dustin Dannenhauer, Héctor Muñoz-AvilaIJCAI 2015

Abstract

Goal-Driven Autonomy (GDA) agents operating in dynamic environments must continuously monitor their progress to determine if their current plans remain viable. Traditional monitoring strategies often fall into two extremes: "Immediate Expectations," which only check the next action's preconditions, and "State Expectations," which compare the entire world state against a predicted model. This paper introduces Informed Expectations, a middle-ground approach integrated into Hierarchical Task Network (HTN) planning. By monitoring only the environmental variables relevant to the plan's trajectory, Informed Expectations avoid the short-sightedness of immediate checks and the over-sensitivity of full-state monitoring. Experimental results in the Marsworld and Arsonist domains demonstrate that Informed Expectations achieve high plan completion rates with significantly lower execution costs than state-based approaches.

Introduction

In autonomous systems, Goal-Driven Autonomy (GDA) provides a framework for agents to monitor their execution, detect discrepancies, and reason about goal changes. A critical challenge in GDA is determining what environmental changes constitute a meaningful discrepancy.

Current approaches typically employ one of two strategies:

  1. Immediate Expectations: These agents only verify the preconditions of the very next action. While computationally inexpensive, they are "blind" to future plan failures caused by environmental changes that do not affect the immediate step.
  2. State Expectations: These agents compare the entire current state of the world against a predicted state. While robust, this approach is often "paranoid," triggering unnecessary re-planning for environmental changes that are irrelevant to the agent's specific goals.

This paper proposes Informed Expectations to resolve this "Goldilocks" problem, providing a monitoring strategy that is both safe and efficient.

The Informed Expectations Approach

The authors implement Informed Expectations within a Hierarchical Task Network (HTN) planner (PyHop). The core innovation lies in using the plan's hierarchical structure to filter environmental noise.

Formalization and Algorithm

Instead of checking the entire state, the agent elicits expectations based on the plan trajectory. It identifies which state variables are modified or required by the sequence of tasks in the hierarchy. The authors introduce the SEEK-EX algorithm, an extension of the standard SHOP planning algorithm. As the planner decomposes high-level goals into primitive actions, it builds a dictionary of informed expectations that track the cumulative requirements and effects of the plan as it evolves.

This approach allows variables to remain "undefined" if they are not relevant to the task, preventing the agent from flagging irrelevant environmental changes as errors.

Experimental Evaluation

The approach was evaluated using two benchmarks:

  • Marsworld: A grid-based navigation domain where agents must contend with "mud" (slowing movement) and "magnetic radiation clouds" (deactivating beacons).
  • Arsonist Domain: A variant of Blocksworld where an "arsonist" randomly sets blocks on fire, preventing the agent from stacking them until the fire is extinguished.

Results and Discussion

The experiments compared three agent types: Immediate, State, and Informed.

Figure 1: Cumulative execution costs for the Marsworld and Arsonist domains. The Informed agent (green) consistently outperforms the State agent (blue) by avoiding unnecessary re-planning while maintaining higher success rates than the Immediate agent (red).

In Marsworld, the State agent incurred high costs due to "false anomalies" triggered by irrelevant mud. The Immediate agent failed frequently because it did not notice when beacons were deactivated by clouds. The Informed agent matched the State agent's success rate but with significantly lower execution costs.

Figure 4: Plan completion rates as environmental volatility increases. In the Arsonist domain (b), the Immediate agent's performance drops sharply as fire probability increases, whereas Informed and State agents remain robust.

In the Arsonist Domain, the Informed agent successfully detected future failures (e.g., a block needed later catching fire) and triggered goal-reasoning to extinguish the fire, whereas the Immediate agent only noticed the fire when it attempted to use the block, often too late to recover efficiently.

Conclusion

Informed Expectations provide a scalable and robust monitoring strategy for GDA agents. By focusing on plan-relevant variables, agents can ignore environmental noise without sacrificing safety. Future work includes integrating these expectations with learning systems to improve discrepancy explanation and refining HTN domain models based on observed violations.

Summary updated

BibTeX
@inproceedings{dannenhauer2015raising,
  author = {Dustin Dannenhauer and Héctor Muñoz-Avila},
  title = {{Raising Expectations in GDA Agents Acting in Dynamic Environments}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2015},
}
Figure