The Art of Artificial Intelligence: Themes and Case Studies of Knowledge Engineering

This paper, "The Art of Artificial Intelligence: Themes and Case Studies of Knowledge Engineering" by Edward A. Feigenbaum, delves into the burgeoning field of knowledge engineering, focusing on how to imbue AI systems with expert-level knowledge. It argues that the true power of AI lies not in sophisticated inference engines, but in the richness and accessibility of the knowledge they possess, illustrated through several pioneering case studies.

TL;DR

This paper explores the art of knowledge engineering, demonstrating how expert knowledge can be captured and utilized to build powerful AI systems. Through case studies like PUFF, DENDRAL, and MYCIN, it highlights the crucial role of domain-specific knowledge and the "generation-and-test" paradigm in achieving expert-level performance in complex tasks.

What Problem Does This Paper Address?

The paper addresses the fundamental challenge in Artificial Intelligence: how to create intelligent agents that can perform complex tasks at an expert level. It posits that the traditional AI focus on general problem-solving techniques and powerful inference engines is insufficient. Instead, the key lies in acquiring, representing, and effectively using vast amounts of domain-specific knowledge. This "knowledge-based" approach aims to bridge the gap between human expertise and machine capability, enabling AI to tackle real-world problems in areas like medical diagnosis and scientific discovery.

Methods

The paper emphasizes several core methodologies:

  • Knowledge Engineering: The central theme is the practice of extracting, representing, and utilizing expert knowledge. This involves intensive collaboration between knowledge engineers and domain experts.
  • Generation-and-Test: This is presented as a ubiquitous framework in AI research. It involves generating potential solutions or hypotheses and then testing them against data or criteria. This is illustrated in systems like DENDRAL and AM.
  • Situation => Action Rules: This is the primary representation for expert knowledge. Rules capture specific conditions (situation) that lead to certain actions or conclusions. This representation is highlighted for its modularity and ease of modification.
  • Backward Chaining Inference: Systems like PUFF and MYCIN employ backward chaining to find a line of reasoning from a goal to the available data.
  • Explanation Capabilities: A critical aspect is the ability of the system to explain its reasoning process to the user, fostering trust and aiding in debugging and knowledge acquisition.

Datasets

The paper does not explicitly list or introduce new benchmark datasets in the modern sense. Instead, it describes the use of:

  • Patient Data: For the PUFF system, real patient pulmonary function test data and medical records were used.
  • Chemical Structures and Mass Spectra: For DENDRAL and META-DENDRAL, data from known molecular structures and their corresponding mass spectra were used.
  • Medical Cases: For MYCIN, clinical cases of blood infections and meningitis were used for diagnosis and treatment recommendation.
  • Simulated Data: For SU/X, signal data generated by a simulation program was used.

The focus is on using real-world or representative data to extract and validate expert knowledge, rather than on standardized datasets for performance comparison.

Key Insights and Ideas

  • Knowledge is Power: The paper strongly advocates that the expertise of an AI system stems primarily from the specialist knowledge it possesses, rather than the sophistication of its inference engine.
  • The "Intelligent Agent" Viewpoint: AI research should focus on building systems that act as intelligent agents, capable of understanding and acting on human intent, filling in vagueness, and making specific abstractions.
  • Expertise is Tacit: Experts often possess knowledge they are not consciously aware of ("know more than they are aware of knowing"). Knowledge engineering aims to uncover this tacit knowledge.
  • Modularity and Extensibility: The "Situation => Action" rule representation facilitates modularity, making it easier to add, remove, or modify knowledge as the expert's understanding evolves.
  • Explanation is Crucial: The ability to explain reasoning is not just a philosophical nicety but an engineering necessity for user acceptance, debugging, and knowledge acquisition.
  • The Art of Engineering: Knowledge engineering is presented as an art, requiring skilled craftsmanship to build complex systems that represent and reason with world knowledge.

Key Results

The paper showcases several systems achieving expert-level performance:

  • PUFF: Achieved expert-level performance in diagnosing pulmonary function disorders, with a high degree of agreement (90-100%) with expert diagnoses, developed in a remarkably short time (less than 50 hours of expert interaction).
  • DENDRAL: Demonstrated success in inferring chemical structures from mass spectrometry data, rivaling expert human performance in specific molecular families, and often being faster and more accurate.
  • MYCIN: Showed strong performance in diagnosing blood infections and meningitis, with a majority of expert judges agreeing with its decisions in 90% of cases.
  • META-DENDRAL: Successfully recreated rule sets comparable in quality to those acquired from human experts, even discovering new rules for complex molecular families.
  • SU/X: Achieved expert-level performance in signal understanding tasks using simulated data.

Limitations and Future Work

  • Hardware and Software Limitations: The paper notes the lack of adequate and appropriate computer hardware and the difficulty in exporting systems due to the lack of properly-sized and packaged software.
  • Cumulation of Techniques: A chronic absence of widely usable software packages for AI techniques is identified.
  • Shortage of Trained Engineers: A shortage of trained knowledge engineers is highlighted as a significant bottleneck.
  • Funding Challenges: Obtaining and sustaining funding for interesting knowledge engineering projects is a difficulty.
  • Future Work: The paper suggests that future research will focus on the general concerns of knowledge engineers, including defining "application" of AI, selecting appropriate applications, finding and fascinating experts, designing systems that "think the way I do," sustaining attention through feedback, focusing on data and specific problems, and providing ways to express uncertainty of expert knowledge. The development of more sophisticated explanation capabilities and the integration of diverse knowledge sources remain key areas.

Summary updated

BibTeX
@inproceedings{1977art,
  title = {{The Art of Artificial Intelligence: Themes and Case Studies of Knowledge Engineering}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1977},
}
Figure