Informed Expectations to Guide GDA Agents in Partially Observable Environments

Dustin Dannenhauer, Hector Muñoz-Avila, Michael T. CoxIJCAI 2016

Introduction

Goal-Driven Autonomy (GDA) agents are designed to navigate complex environments by monitoring for "discrepancies"—unexpected changes that signal a need to revise current plans. However, traditional GDA research often operates under the assumption that sensing is free and that agents possess sufficient information to plan from start to finish. In real-world applications, sensors consume critical resources like battery power and bandwidth, and environments are frequently partially observable. This paper introduces Informed Expectations, a method for agents to balance the necessity of "looking before they leap" with the actual cost of sensing.

The Guiding Sensing Problem

The authors formalize the "Guiding Sensing Problem," distinguishing between planning actions (e.g., moving) and sensing actions (e.g., checking a sensor), which carry specific costs. The goal is to find a "Goldilocks" zone: sensing enough to detect critical environmental changes without wasting resources on irrelevant data.

To address this, four strategies for managing expectations were evaluated:

  • No Expectations: Sensing only the preconditions of the immediate next action.
  • Immediate Expectations: Sensing preconditions and expected effects of the next action.
  • Eager Expectations: Checking every known condition about the world at every step.
  • Informed Expectations: A novel approach where the agent tracks all conditions validated by previous actions in the current plan. It only senses these "relevant" conditions to ensure the path already taken remains valid.

Experimental Evaluation

The strategies were tested in two simulated environments: Marsworld, a grid-based beacon activation task, and Blockscraft, a tower-building domain where external agents can disrupt progress. In both domains, environmental events (like fires going out or blocks being moved) created discrepancies that agents needed to detect.

Performance and Efficiency

As shown in the results, agents utilizing Informed and Eager expectations achieved nearly 100% of their goals. In contrast, "None" and "Immediate" strategies frequently failed because they were "forgetful"—they did not notice when a previously satisfied condition (like an activated beacon) was reversed by an external event.

Comparison of goal achievement, sensing cost, and actions executed across agent strategies.

While both Informed and Eager strategies were successful, Informed Expectations proved significantly more efficient. By focusing only on relevant conditions—essentially asking, "Is everything I've done so far still standing?"—the Informed agent achieved the same safety as the Eager agent at a fraction of the sensing cost.

Scalability in Dynamic Environments

The advantage of Informed Expectations becomes even more pronounced as environmental volatility increases. In scenarios with a higher frequency of external events (discrepancy rates), the cost gap between Informed and Eager sensing widens.

Sensing cost as a function of external event frequency (discrepancy rate).

As environmental chaos increases, Eager agents must spend more resources to maintain a complete world model. Informed agents, however, maintain a relatively stable and lower sensing cost because they only monitor the specific subset of the environment that impacts their current plan's validity.

Conclusion

Informed Expectations provide a robust middle ground for GDA agents in partially observable, dynamic worlds. By maintaining a "memory" of relevant plan conditions, these agents achieve high success rates while minimizing resource expenditure. Future work may explore dynamic sensing frequencies and goal-regression expectations to further optimize performance in even more complex domains.

Summary updated

BibTeX
@inproceedings{dannenhauer2016informed,
  author = {Dustin Dannenhauer and Hector Muñoz-Avila and Michael T. Cox},
  title = {{Informed Expectations to Guide GDA Agents in Partially Observable Environments}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure