Maintaining Evolving Domain Models

Daniel Bryce, J. Benton, Michael W. BoldtIJCAI 2016

Abstract

In automated planning, domain models are typically treated as static, immutable representations of the world. However, real-world operations frequently encounter "model drift," where the AI's internal model diverges from the actual ground truth due to sensor failures, procedural changes, or human intervention. This paper introduces Marshal, a model maintenance system that utilizes Bayesian filtering and active learning to track and update evolving domain models. By observing plan execution and proactively querying users, Marshal maintains synchronization between the planner's knowledge and the dynamic operational environment.

Introduction

The "rules of the game" in AI planning—the domain model—are rarely as fixed as research benchmarks suggest. In complex environments like the International Space Station (ISS), reality is messy. When a domain model becomes outdated, the AI continues to generate plans based on invalid assumptions, leading to persistent failures. While significant research has focused on initial knowledge acquisition, the problem of model maintenance—keeping a deployed model accurate over time—remains under-addressed. Manual updates are costly and error-prone, creating a need for automated systems that can anticipate and adapt to "bit rot" in PDDL (Planning Domain Definition Language) files.

The Model Maintenance Problem (MMP)

The authors formalize this challenge as the Model Maintenance Problem (MMP). They model the "true" domain as a hidden state in a stochastic process, akin to a Hidden Markov Model. The goal is to maintain a belief state over possible models based on observations of plan outcomes (success or failure) and direct feedback from human experts.

The Marshal System

Marshal addresses the MMP through several key technical components:

  • Particle Filtering: The system maintains a belief state represented by hundreds of model variations (particles). These particles are re-weighted as evidence is gathered from the environment.
  • Active Learning and Query Generation: Rather than passively waiting for data, Marshal identifies model features most relevant to explaining plan outcomes. It generates targeted questions for the user (e.g., "Is X a precondition for action Y?") to efficiently prune the model search space.
  • Drift Heuristics: To constrain the search, Marshal employs "well-formed drift" heuristics. For example, it might assume that if a property is a precondition for an action, it is likely also a delete effect.
  • Generalization: Marshal can generalize learned rules across action schemas, applying a discovery about one specific entity (e.g., a specific robot) to all entities of the same type.

Experimental Evaluation

The researchers evaluated Marshal using four domains from the 2014 International Planning Competition (IPC-2014): Parking, Spanner, Transport, and Floortile. They simulated a "drifting" environment where the ground-truth model was intentionally modified six times during a session to test Marshal's recovery speed and accuracy.

Results

The evaluation focused on two primary metrics: model error (the structural difference between the belief and the truth) and testing error (the discrepancy between predicted and actual plan validity).

Average model error across IPC-2014 domains for varying particle counts and drift heuristics.

The "Well-formed Generalization" (WG) method consistently achieved the lowest model error across all test domains. Furthermore, the system demonstrated clear scaling benefits; increasing the number of particles from 128 to 1024 significantly reduced the average error, as shown in Figure 1.

Average testing error showing the impact of particle scaling and query strategies on plan validity prediction.

Testing error also decreased as the number of particles and queries per plan increased. Marshal proved capable of hypothesizing explanations for failures and repairing models with minimal user intervention, effectively shifting the human's role from a manual debugger to a high-level domain expert reviewer.

Conclusion

Marshal demonstrates that treating domain models as dynamic entities is essential for long-term AI deployment. By combining Bayesian filtering with common-sense drift heuristics, the system can effectively track model evolution. Future work will extend these capabilities to handle more complex representations, including temporal and resource constraints, moving toward a full mixed-initiative planning framework where humans and AI collaborate to refine both plans and the rules that govern them.

Summary updated

BibTeX
@inproceedings{bryce2016maintaining,
  author = {Daniel Bryce and J. Benton and Michael W. Boldt},
  title = {{Maintaining Evolving Domain Models}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure