Introduction
In the early 1980s, the field of Artificial Intelligence faced a significant hurdle known as the "knowledge acquisition bottleneck." While expert systems were demonstrating success in specialized domains, their development relied heavily on "knowledge engineers" who manually translated human expertise into computer code. This process was slow, error-prone, and often failed to capture the nuanced strategic thinking of the expert.
Presented at AAAI-83, Benjamin and Harrison’s "A Production System for Learning Plans From an Expert" proposed a novel solution. Rather than hand-coding every decision, the authors developed a system capable of "learning" strategic control information by observing and interacting with a human expert. By extending the OPS5 production system, the researchers created a framework that could generalize specific actions into broader, reusable plans.
The Challenge: Logic vs. Control
The authors identified a fundamental distinction in expert knowledge: the "logic" of the domain versus the "control" of the problem-solving process. Logic refers to the underlying facts and rules (e.g., "if X is true, then Y is a possible action"). Control refers to the strategy used to navigate those rules (e.g., "given five possible actions, which one should be performed first?").
Traditional production systems were often rigid. If a situation did not perfectly match a pre-defined rule, the system would stall. Furthermore, encoding the "control" expertise was notoriously difficult because experts often apply strategies intuitively without explicitly stating their underlying plan. Benjamin and Harrison aimed to automate the extraction of this control expertise, allowing a system to plan and generalize its behavior dynamically.
System Architecture
The proposed system was built upon OPS5, a prominent production system language of the era. To move beyond simple rule-matching, the authors introduced several critical components:
Meta-productions
The system modified the standard OPS5 architecture to support "meta-rules"—rules that reason about other rules. These meta-productions inspect the "conflict set" (the collection of all rules currently eligible to fire) and apply strategic criteria to select the most appropriate action. This allowed the system to possess a level of self-reflection regarding its own decision-making process.
PLEX (Planning Expert)
PLEX served as the interface between the system and the human trainer. When an expert performed an action, PLEX did not just record the step; it queried the expert for the underlying subgoal structure. This interaction resulted in the construction of a hierarchical goal tree, utilizing nodes such as SEQUENCE, REPEAT, AND, and OR to represent the logic of the plan.
GENEX (Generalization Expert)
Once a specific sequence of actions was captured, GENEX was responsible for generalization. It analyzed the specific trace provided by the expert and transformed it into broader rules. This ensured that the system didn't just memorize a single solution path but learned a strategy applicable to a wider class of similar problems.
Goal Trees in Working Memory
Plans were represented as active data structures within the system's working memory. This allowed the system to track its progress through a complex hierarchy, handle iterative tasks via REPEAT nodes, and implement backtracking mechanisms if a particular branch of a plan failed.
Experimental Demonstration: The Jigsaw Puzzle
To demonstrate the system's capabilities, the authors applied it to a jigsaw puzzle solver. Initially, the system operated with a "nondeterministic" approach, essentially attempting to place pieces at random—a highly inefficient method.
A human trainer then provided a specific strategy, such as "Find a piece with a matching edge, then place it." By observing this specific instance and asking for the "why" behind the actions, the system used PLEX and GENEX to build a generalized goal tree.
Key Insights and Results
The research yielded several significant insights into the nature of automated learning:
- Separation of Concerns: By separating domain logic from strategic control, the learning process became more tractable.
- Interactive Planning: The shift from passive observation to active interaction (asking for subgoals) allowed the system to capture the hierarchical nature of human expertise.
- Efficiency Gains: The primary qualitative result was the transition from exhaustive, slow searches to direct, efficient solution paths. Once the goal productions were learned and generalized, the system could solve new puzzles of the same type without further human intervention.
Limitations and Future Directions
Despite its success as a proof-of-concept, the authors noted several challenges. The "goal structure" space is significantly more complex and sparse than the physical structure of a problem, making it difficult for the system to anticipate every environmental change. Furthermore, in complex domains, the system often had to make decisions based on incomplete information.
Finally, the authors acknowledged that while the jigsaw puzzle provided a clear demonstration, more robust description mechanisms would be required to handle non-probabilistic computational problems, many of which are NP-hard and require even more sophisticated generalization techniques.
Conclusion
Benjamin and Harrison’s work represented a significant step toward "reflective" AI. By giving a production system the tools to observe, question, and generalize expert behavior, they provided a blueprint for overcoming the knowledge acquisition bottleneck, moving the field closer to systems that can truly learn how to think, rather than just what to do.
Summary updated