Introduction
In the late 1980s, the "knowledge engineering bottleneck" stood as a primary obstacle in artificial intelligence. Developing expert systems required human specialists to articulate complex mental models—a task complicated by the fact that experts are often more proficient at execution than explanation. While Machine Learning (ML) offered a path to automate rule extraction from examples, "noise" remained a significant disruptor.
At the time, most research treated noise as simple statistical error to be smoothed over with probability coefficients. In "Noise and Knowledge Acquisition," Manago and Kodratoff argue that this approach is insufficient. They define noise as a multi-faceted structural problem involving unreliable, incomplete, and poorly defined information. The paper proposes a symbolic ML approach that utilizes cross-examination, refined taxonomies, and strategic decision-tree construction to build robust expert systems without sacrificing explainability.
Methodology
The authors employ a hybrid approach integrating several ML tools and knowledge representation techniques:
- Learning Engines (NEDDIE & MAGGY): The system uses NEDDIE, a descendant of ID3, to construct decision trees. MAGGY, a descendant of AGAPE, generalizes clusters of examples into comprehensive concept descriptions.
- Frame-Based Representation: Knowledge is organized into frames with "slots," utilizing procedural "daemons" to manage missing information or default values (e.g., assuming a leaf is green unless specified otherwise).
- Strategic Delaying: To handle unreliable data, the system is designed to perform noisy or costly tests as late as possible in the decision tree, prioritizing more reliable descriptors.
- Cross-Examination: By comparing the domain models provided by different experts, the system identifies descriptors that are inherently noisy or require high skill to identify.
Datasets
The research draws on three primary real-world applications:
- Tomato Plant Pathology: A knowledge base of over 350 rules for diagnosing diseases, developed with the French Institut National de Recherche en Agronomie (INRA).
- Air Traffic Control: A large-scale application analyzing radar data to understand expert reactions to flight patterns.
- Legal Cases: A dataset of 57 court cases regarding the legal duties of city mayors.
Key Insights: Redefining Noise
The core contribution of this work is the categorization of noise into three distinct types, each requiring a specific treatment:
- Unreliable Noise: Features that are hard to observe or costly to test.
- Erroneous Noise: Simple human error in data entry or observation.
- Incomplete Noise: Missing examples or descriptors that create a "gap" in the version space.
The authors argue for learning "full descriptions"—the essential characteristics of a concept—rather than just consultation rules. This allows the system to dynamically generate alternative reasoning paths if a user cannot answer a specific question. Furthermore, they suggest that "noise" is often a failure of the description language itself. Refining taxonomies (e.g., introducing "translucent" to a color hierarchy) can resolve classification errors that numeric coefficients cannot.
Results and Discussion
The paper provides a qualitative framework for robust Knowledge Acquisition (KA). Key successes include:
- Structural Robustness: By delaying unreliable tests, systems maintained high performance even with noisy user input.
- Symbolic Conflict Resolution: The authors demonstrated that modifying frame hierarchies could resolve classification errors (e.g., distinguishing "Sunburn" from "Narcosis") where statistical methods failed.
- Expert Alignment: Cross-examination effectively filtered subjective terms into objective descriptors.
However, limitations remain. The authors note they lack methods for detecting noise within background knowledge (axioms and constraints). Additionally, while the system detects inconsistencies, it still relies on human experts for final resolution. Future work points toward more adaptive models capable of handling shifting statistical data in dynamic environments.
Summary updated