Learning Unknown Event Models

Matthew Molineaux, David W. AhaAAAI 2014 · DOI

Abstract

In classical AI planning, agents often operate under the assumption of a complete and perfect domain model. However, real-world environments frequently present unexpected exogenous events that deviate from these models. This paper introduces FOOLMETWICE, a Goal-Driven Autonomy (GDA) agent that learns to model such unknown events by treating "surprises" as supervised learning signals. By combining automated explanation generation with relational learning (FOIL-PS), the agent can hypothesize, test, and refine models of exogenous events. Experimental results in simulated domains demonstrate that FOOLMETWICE significantly reduces plan execution costs, achieving near-optimal performance within a few training trials.

Introduction: The Challenge of the Unexpected

Traditional AI planning systems typically rely on a "perfect" map of environmental dynamics. In reality, unexpected events—such as a robot's wheels slipping in unmapped mud or a satellite's lens cracking due to an unmodeled solar flare—can lead to plan failure. These "exogenous events" cause a discrepancy between an agent's expectations and its actual observations.

Most agents respond to such surprises by failing or reverting to safe, sub-optimal modes. FOOLMETWICE addresses the challenge of situated learning: how an agent can detect these surprises, explain their likely causes, and learn a formal model of the new event to improve future planning.

Methods: The Surprise-Explain-Learn Loop

FOOLMETWICE extends the Goal-Driven Autonomy (GDA) framework by implementing a continuous loop for model refinement.

The FOOLMETWICE architecture integrates a Controller with specialized modules for discrepancy detection, explanation generation, and goal management.

The workflow consists of four primary stages:

  1. Discrepancy Detection: The agent compares its expected state (derived from its current model) with actual observations. Any mismatch flags a "discrepancy."
  2. Recognizing Unknown Events: Using the DISCOVERHISTORY algorithm, the agent searches for the most plausible explanation for the discrepancy. If known events cannot explain the data, the agent applies "inconsistency patches"—placeholders indicating that an unknown event with specific effects occurred.
  3. Generalizing Preconditions (FOIL-PS): To understand why an event happened, the authors adapted the First-Order Inductive Learner (FOIL) into FOIL-PS (Projected States). This module analyzes states immediately preceding the surprise (positive examples) and states where it did not occur (negative examples) to induce logical rules for the event's preconditions.
  4. Model Modification: The induced rule is converted into a formal event model and integrated into the agent's knowledge base. If subsequent predictions fail, the agent can "abandon" the model and re-learn.

Experimental Evaluation

The agent was evaluated in two deterministic, partially observable simulated domains:

  • Satellites: A satellite must capture images, but photographing "excessively bright objects" (a hidden property) breaks the camera. The agent must learn the relationship between calibration, lens flares, and camera failure.
  • MudWorld: A robot navigating a grid encounters unobservable muddy cells that halve its speed. The robot must learn to associate "nearby mud" sensor readings with the hidden mud and its impact on movement.

Key Results

FOOLMETWICE demonstrated rapid adaptation in both test environments. Within just five training trials, the agent achieved a performance gain of more than 90% relative to an "optimal" agent equipped with a perfect prior model.

Performance comparison across training scenarios shows FOOLMETWICE (Learned) rapidly approaching the efficiency of an agent with a complete model.

Even after a single training scenario, the agent's performance significantly outperformed the non-learning baseline. Furthermore, the agent successfully demonstrated error correction, abandoning incorrect models when they led to new discrepancies and eventually converging on accurate representations of environmental dynamics.

Discussion and Future Work

While FOOLMETWICE shows promise, several limitations remain. The current implementation assumes a deterministic world and struggles with probabilistic events or noisy sensor data. Additionally, the agent is currently limited to discrete changes; future work will explore PDDL+ to model continuous processes like fuel consumption or temperature fluctuations.

Another area for development is "active transfer learning," where models learned in one environment are used to accelerate understanding in similar, novel domains. This would further enhance the agent's ability to operate autonomously in complex, unpredictable real-world settings.

Summary updated

BibTeX
@inproceedings{molineaux2014learning,
  author = {Matthew Molineaux and David W. Aha},
  title = {{Learning Unknown Event Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2014},
  doi = {10.1609/aaai.v28i1.8751},
}
Figure