Introduction
In classical AI planning, agents navigate environments using "operators"—predefined actions characterized by preconditions (what must be true to act) and effects (what changes after the action). Traditionally, these operators are hand-coded by domain experts, creating a significant "knowledge engineering bottleneck." This manual process is tedious, prone to errors, and scales poorly to complex real-world environments. This paper presents a framework designed to automate the acquisition of these operators by observing expert behavior and refining knowledge through iterative practice.
The Integrated Learning Framework
The proposed system is built upon the PRODIGY architecture, integrating four primary modules to facilitate autonomous learning.
As shown in the figure above, the architecture creates a feedback loop between observation, planning, and execution. The Observation Module monitors an expert agent's actions, while the Learning Module processes these observations to generate or refine operators. These operators are then utilized by the Planning Module to generate tentative plans, which the Plan Execution Module attempts to carry out in the environment. Any failures encountered during execution are fed back into the learning cycle, ensuring continuous improvement.
The Learning Lifecycle
The framework follows a four-stage lifecycle to move from zero knowledge to functional expertise:
- Observation: The system records an expert's action sequences, noting the state of the environment before (pre-state) and after (post-state) each action.
- Initial Induction: Upon encountering a new action, the system induces a "conservative" operator. It assumes every condition currently true in the environment is a necessary precondition. Effects are defined by the delta between the pre-state and post-state.
- Refinement via Generalization: By observing the same action in different contexts, the system compares new pre-states to its existing rules. If a previously assumed precondition is absent but the action remains successful, the system prunes that condition, gradually generalizing the operator.
- Practice and Plan Repair: The agent attempts to solve problems independently. When a plan fails during execution, the Plan Execution Module provides a trace to the Learning Module. The system then repairs the plan and updates the operator based on the specific cause of failure, effectively learning from its own mistakes.
Experimental Evaluation
The framework was evaluated in two simulated environments: the Extended-STRIPS domain (a classic blocks-world scenario) and the Process Planning domain (involving complex manufacturing tasks). The system successfully demonstrated the ability to refine partially incorrect or incomplete operators into functional ones through its integrated learning-planning-repair cycle. Rather than using static datasets, the system generated its own training data through simulated expert traces and self-directed practice.
Key Insights and Limitations
A key insight of this work is the "specific-to-general" inductive process. By starting with highly specific operators and generalizing only when evidence permits, the system avoids the risks associated with overly broad initial assumptions. Furthermore, the integration of learning with execution ensures that the agent's world model remains dynamic and responsive to real-world feedback.
However, the current framework operates under several simplifying assumptions:
- Full Observability: The agent is assumed to have perfect information about the environment state at all times.
- Determinism: The environment is assumed to be noise-free and predictable, with perfect sensors.
- Conjunctive Logic: The system currently only learns "AND" rules for preconditions, lacking support for "OR" logic or conditional effects.
Future research aims to extend this algorithm to handle noisy, partially observable environments, bringing it closer to real-world applicability.
Summary updated