A KNACK for Knowledge Acquisition

Georg Klinker, Casey Boyd, Serge Genetet, John P. McDermottAAAI 1987

Introduction

In the late 1980s, the development of expert systems faced a significant hurdle known as the "knowledge acquisition bottleneck." Building these systems typically required a laborious collaboration between a domain expert and a knowledge engineer. This process was often slow, expensive, and prone to miscommunication. KNACK was introduced as an automated tool designed to bypass this bottleneck, allowing domain experts to build specialized expert systems—termed "WRINGERs"—without the direct intervention of a programmer.

The Knowledge Acquisition Bottleneck

The traditional approach to building expert systems involved months of interviews where a knowledge engineer attempted to translate an expert's intuition into formal code. KNACK addresses this by targeting the domain of electromechanical design evaluation, specifically nuclear hardening. The system "interviews" the expert, analyzes sample reports, and automatically generates a functioning expert system capable of evaluating new designs and producing technical documentation.

Methods: A Report-Driven Approach

KNACK’s methodology is fundamentally "report-driven." Rather than requiring experts to write complex IF-THEN rules, it leverages their existing workflow by using sample technical reports as the primary knowledge source.

Conceptual Modeling and Heuristic Generalization

The process begins with Conceptual Modeling, where KNACK establishes a high-level understanding of the evaluation task. It builds a semantic network representing the domain's entities and their relationships. For instance, it understands that an enclosure comprises apertures and that environments produce specific conditions.

A fragment of the conceptual model used by KNACK to represent domain knowledge.

Following this, KNACK employs Heuristic Generalization. When an expert provides a sample report, the system identifies specific instances (e.g., "S-280C enclosure") and replaces them with variables (e.g., <ENCLOSURE.NAME>). This "variabilization" allows the system to generalize from a single example to a broader class of problems.

Strategy Elicitation and the Critique Loop

For every identified variable, KNACK prompts the expert to define a strategy for finding its value—whether through user input, database lookups, or mathematical formulas. To verify its understanding, KNACK enters a Critique Loop. It generates "mock" reports based on various data combinations. When the expert corrects errors in these mock reports, KNACK uses that feedback to refine its internal rules, which are written in the OPS5 production language. Finally, Knowledge Verification checks ensure there are no gaps in the logic, such as unused concepts.

Evaluation and Results

The efficacy of KNACK was demonstrated through the creation of two distinct expert systems in the nuclear hardening domain:

  • PROGRAM PLAN: A management-level report generator built in just one person-week.
  • DESIGN PARAMETERS REPORT: A detailed technical evaluation system built in three person-weeks.

Compared to the months or years typically required for manual expert system development in the 1980s, these results represent a massive leap in productivity. Analysis showed that approximately 7.5% to 8.7% of the words in generated reports were "active" values derived from the system's logic, confirming that the tool was performing substantive analytical work rather than simple template filling.

Discussion and Future Directions

While highly successful, KNACK faced limitations. Its parsing relied on relatively simple heuristics rather than deep natural language processing, occasionally leading to incorrect guesses. Furthermore, the generated systems were heavily dependent on the quality and breadth of the initial sample reports.

Despite these challenges, KNACK proved that domain experts without AI backgrounds could successfully maintain and update complex knowledge bases, effectively bridging the gap between human expertise and automated reasoning.

Summary updated

BibTeX
@inproceedings{klinker1987knack,
  author = {Georg Klinker and Casey Boyd and Serge Genetet and John P. McDermott},
  title = {{A KNACK for Knowledge Acquisition}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1987},
}

Loading related content...

Figure