Introduction
In the early 1980s, the field of Artificial Intelligence was transitioning from academic curiosity to industrial application. One of the most significant challenges of this era was the "Expert System"—a software paradigm designed to capture human expertise in specialized domains. In her 1982 paper presented at the AAAI proceedings, Susan Ennis of Amoco Production Company provided a rare, pragmatic evaluation of these tools. By attempting to solve a concrete geological problem—identifying minerals from x-ray diffraction (XRD) spectra—Ennis highlighted the friction between academic AI research and the requirements of industrial production.
The Problem: Mineral Identification from XRD
The identification of minerals in rock samples is a cornerstone of geological analysis. X-ray diffraction produces spectra where peaks represent characteristic atomic spacings. In a perfect world, identifying a mineral would be a simple matter of pattern matching. However, real-world laboratory data is rarely clean. Peak intensities vary based on sample preparation, and peaks from different minerals frequently overlap or coincide.
Human experts solve this through an iterative "determine-and-subtract" process. A mineralogist identifies the most prominent mineral, estimates its contribution, subtracts that contribution from the spectrum, and then repeats the process with the remaining data. Amoco sought to determine if the burgeoning "Expert System" shells could automate this heuristic process and, crucially, if a non-programmer (a mineralogist) could maintain the resulting knowledge base.
Comparative Methodology
Ennis conducted a comparative study by implementing the mineral identification logic across five distinct software environments:
- EXPERT: A representation language specifically designed for consultation models.
- LISP: A custom-built system developed from scratch using the LISP programming language.
- UNITS: A frame-based system utilizing inheritance and structured knowledge representation.
- EMYCIN: The "Essential MYCIN" shell, a domain-independent version of the famous MYCIN medical diagnosis system.
- OPS-5: A production system language based on "if-then" rules.
The goal was to recognize approximately 25 minerals, covering 90% of sedimentary rock compositions, using the Joint Committee on Powder Diffraction Standards (JCPDS) as a reference. The systems were tested against several hundred rock samples per week, with results compared against the findings of expert mineralogists.
Key Findings: The Gap Between Theory and Practice
The study revealed several critical insights into the limitations of early AI tools when applied to industrial physics problems:
The Separation of Control and Domain Logic
A recurring theme was the difficulty of separating "domain rules" (geological facts) from "control rules" (computational instructions like looping or branching). In systems like OPS-5, these were often intertwined. This created a significant barrier: if a mineralogist cannot read or update the system without understanding the underlying control logic, the promise of a "maintainable knowledge base" is lost.
Tool Rigidity
Many tools proved too rigid for the iterative nature of the XRD problem. For instance, EXPERT was eventually abandoned because it did not allow a parameter to be changed once it was set. In an iterative subtraction process, where values must be updated as minerals are identified, this was a fatal architectural flaw.
The Necessity of Customization
None of the off-the-shelf tools worked "out of the box." EMYCIN, for example, required the addition of custom iteration functions to handle the repetitive nature of the task. The "Knowledge Engineer" frequently had to step in to add custom data types, such as arrays for spectral data, which the base shells were not designed to handle.
Results and Time-to-Build
The study provided a unique look at the development velocity of early AI projects:
- OPS-5: Fastest to prototype (1.5 weeks for 3 minerals) but the most difficult to maintain.
- EMYCIN: Required 3 weeks for 4 minerals, following the addition of custom functions.
- UNITS: Took 1 month to handle 4 minerals.
- LISP: Required 1.5 months for the core engine and an additional 2 months for a user-friendly interface.
Despite the longer development time, the custom LISP implementation was the most successful. It allowed mineralogists to edit rules using English-like syntax and see immediate results, leading to the highest level of user satisfaction and practical utility.
Conclusion
Ennis’s evaluation served as a sobering reality check for the AI community. While expert system shells offered a framework for rapid prototyping, they often lacked the "hardening" and flexibility required for production environments. The tools were frequently more rigid than the problems they were meant to solve.
Ennis concluded that the most effective use of these shells might be as a means to generate specifications for a custom system, rather than serving as the final product itself. Her work remains a foundational reminder that the bridge between a research tool and an industrial solution is built on flexibility, robust user interfaces, and the clear separation of domain expertise from programming logic.
Summary updated