Combining Knowledge Acquisition and Machine Learning to Control Dynamic Systems

IJCAI 1997

Sure, here's a blog-style summary of the paper "Combining Knowledge Acquisition and Machine Learning to Control Dynamic Systems":

TL;DR

This paper introduces a novel interactive method that merges knowledge acquisition and machine learning to build controllers for dynamic systems. By combining an expert operator's explicit knowledge with automatically learned rules from their behavior, the system can create more robust and transparent controllers, demonstrated through a flight simulator.

What Problem Does This Paper Address?

Building controllers for complex dynamic systems, especially those requiring real-time, intuitive human-like control (like flying an aircraft), is challenging with traditional methods. While human operators excel at these tasks, their expertise is often subcognitive and difficult to articulate explicitly. Previous attempts at "behavioral cloning" (learning from expert demonstrations) often resulted in opaque "black box" controllers. This paper addresses the gap of creating controllers that are both effective and understandable, by leveraging both explicit expert knowledge and implicit learned behavior.

Methods

The core of the paper is the introduction of a hybrid approach combining two techniques:

  • Dynamic Ripple Down Rules (DRDR): This is an interactive knowledge acquisition method. It starts with a simple rule and interactively refines it by adding exception rules when a rule fails. The system helps the expert by comparing the current failing case with the "cornerstone case" (the original case that led to the rule) to suggest relevant differences for new conditions. DRDR is enhanced with features like multiple knowledge bases (one for each control action) and a graphical interface tailored for pilots.
  • Learning Dynamic Ripple Down Rules (LDRDR): This is the machine learning component. It automatically generates rules from logged flight data. The process involves:
  • Data Preparation: Segmenting flight data into stages, discretizing control values, and eliminating redundant intermediate values.
  • Attribute Prioritization: Creating separate input files for each control action (elevators, ailerons, flaps, throttle) and maintaining a priority list of attributes that are most influential in predicting control actions.
  • Rule Generation: LDRDR iterates through the prioritized attributes, identifying changes in their direction. It creates new tests based on these changes and the attribute's current value. These tests are used to form new rules, which are then added to the DRDR knowledge base.

The system integrates these by allowing the expert to use DRDR for parts of the flight they can articulate, and LDRDR for more complex, subcognitive maneuvers. The generated rules are then tested by running the flight simulator in an "autopilot mode" where the system's control is dictated by these rules.

Datasets

The paper uses data generated from a modified Silicon Graphics flight simulator, specifically a Cessna 150. The "dataset" is essentially the logged flight traces of human pilots performing a predefined flight plan. This data is collected during experimental sessions with volunteers. The paper does not introduce a new, publicly available benchmark dataset but rather uses the simulator itself as the environment for generating experimental data.

Key Insights and Ideas

  • Synergy of Explicit and Implicit Knowledge: The most significant insight is that combining an expert's explicit, rule-based knowledge (DRDR) with machine learning from their implicit behavior (LDRDR) leads to more effective and understandable controllers than either method alone.
  • Interactive Refinement: The interactive nature of DRDR, where the expert is guided by the system to refine rules, is crucial for capturing nuanced expertise that might otherwise be missed.
  • Subcognitive Skill Handling: LDRDR is vital for handling those aspects of control that are subcognitive and difficult for experts to verbalize, effectively "cloning" the learned behavior.
  • Pilot-Centric Interface: The emphasis on a graphical interface that mimics aircraft instruments is key to making the system usable and understandable for domain experts like pilots.
  • Rule Reuse: The observation that pilots reused rules created for earlier flight stages in later stages demonstrates a natural tendency towards generalization and efficiency, which the system can also leverage.

Key Results

  • All three volunteer subjects successfully created a set of rules that could fly the aircraft through the predefined flight plan using the combined DRDR and LDRDR approach.
  • The size of the generated knowledge bases varied significantly between subjects, with some creating more comprehensive rule sets to cover a wider range of potential aircraft states, while others focused on the specific flight plan.
  • The paper demonstrates that the system can generate controllers that are effective, as evidenced by the successful autopilot flights.
  • Qualitatively, the paper highlights that subjects tended to start with DRDR and switch to LDRDR when manual rule creation became difficult, indicating the practical utility of the hybrid approach.

Limitations and Future Work

  • Noise-Free Environment: The current testing environment is noise-free. The authors acknowledge that further work is required to establish whether the technique can handle noisy simulation environments.
  • User Interface Improvement: The authors suggest that the user interface could be further improved. Ideally, pilots should interact with the system through familiar cockpit controls rather than directly seeing or manipulating rules.
  • Rigorous Testing: More rigorous testing is needed to quantitatively measure the relative merits of LDRDR compared to conventional induction programs used in behavioral cloning.
  • Generalizability: While the approach is suggested to be applicable to other domains (like container cranes), further validation in diverse areas is implied.

Summary updated

BibTeX
@inproceedings{1997combining,
  title = {{Combining Knowledge Acquisition and Machine Learning to Control Dynamic Systems}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1997},
}
Figure