The Induction of Fault Diagnosis Systems from Qualitative Models

Danny A. PearceAAAI 1988

Abstract

In the late 1980s, the development of expert systems for fault diagnosis was hindered by the "knowledge engineering bottleneck." This paper proposes a methodology to automate the creation of diagnostic rules by inducing them from qualitative models. By simulating failures in a deep-level causal model and applying rule induction, the authors generated a diagnostic system for a satellite's electrical power subsystem. The resulting system achieved 100% accuracy on test cases, significantly outperforming hand-crafted alternatives while reducing development time and ensuring logical integrity.

Introduction

Traditional expert systems rely on manually encoded "if-then" rules derived from interviews with domain experts. This process is inherently error-prone, difficult to validate, and brittle to hardware changes. D.A. Pearce's 1988 paper addresses these challenges by shifting the focus from manual rule-writing to model-based reasoning. The core idea is to describe how a system's components work (deep knowledge) and allow the computer to derive the diagnostic rules (shallow knowledge) automatically.

Methodology: From Deep Models to Shallow Rules

The proposed methodology involves a three-step pipeline to synthesize diagnostic knowledge bases:

  1. Qualitative Modeling: Instead of numerical differential equations, the system uses symbolic descriptions (e.g., "Normal," "Low," "High") to capture the causal structure of the hardware.
  2. Systematic Simulation: The qualitative model is used to simulate every possible component failure under various operational conditions, creating a comprehensive library of failure examples.
  3. Rule Induction: A version of the AQ algorithm is applied to these examples to induce a compact set of diagnostic rules. This process acts as a form of data compression, transforming thousands of simulation cases into a small, real-time executable rule set.
The process of inducing shallow diagnostic rules from deep qualitative models.

Case Study: Satellite Electrical Power Subsystem

The methodology was applied to the Electrical Power Subsystem (EPS) of an on-station satellite. The training data consisted of 1,464 initial examples generated by failing 61 functional components across different mission phases (eclipse, post-eclipse, and solstice) and payload levels. After filtering for unique, consistent cases, 708 examples remained for rule induction.

Experimental Results and Validation

The automatically synthesized knowledge base (KB) was compared against a hand-crafted expert system built using the commercial shell Envisage. Both systems were tested against 14 failure scenarios from a high-fidelity real-time numerical simulator.

Comparative evaluation of the hand-crafted Envisage KB versus the automatically synthesized KB.

The results demonstrated significant advantages for the automated approach:

  • Accuracy: The induced rules achieved a 100% success rate, compared to 72% for the hand-crafted system.
  • Efficiency: The induction process compressed over 700 examples into just 75 rules, requiring an average of only three attribute tests per diagnosis.
  • Development Speed: The model-based approach required 3 to 4 man-months, whereas the traditional approach took 6 man-months.
  • Integrity: A Knowledge Integrity Checker (KIC) revealed that the hand-crafted system contained type errors and unreachable rules, while the induced system was logically perfect.

Conclusion and Future Work

This research highlights the power of model-based reasoning to overcome the limitations of traditional expert systems. By validating the underlying model rather than individual rules, developers can ensure higher reliability and consistency. However, limitations remain, such as the "single-fault assumption" and the difficulty of handling temporal reasoning. Future work aims to address these by incorporating hierarchical modeling and time-based simulation techniques.

Summary updated

BibTeX
@inproceedings{pearce1988induction,
  author = {Danny A. Pearce},
  title = {{The Induction of Fault Diagnosis Systems from Qualitative Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1988},
}
Figure