Expert systems are traditionally designed to synthesize uncertain facts into coherent conclusions. However, real-world expert domains are often messy, containing evidence that is contradictory or "overconstrained." In this research, J.R. Quinlan critiques the standard "validity-propagating" approach to expert systems and introduces Ponderosa, a system designed to handle internally inconsistent evidence by identifying maximally consistent subsets of data.
The Problem: The Bayesian Trap
Most expert systems, such as the Bayesian system Prospector, assign point probabilities to facts and rules, propagating these values through an inference net. While mathematically sound in a vacuum, these systems struggle with logical conflicts. They often rely on assumptions like conditional independence that rarely hold in complex domains and lack mechanisms to flag when a set of evidence is logically impossible.
When forced to provide categorical (yes/no) answers, Bayesian systems can produce results that violate their own internal rules. Because the math ensures a solution is always found, the resulting "most likely" state may be nonsensical if the underlying evidence is inconsistent.
The Trial Application
To test different reasoning architectures, Quinlan utilized a fictional econometric model consisting of five basic propositions and ten assertions (A1–A10) representing rules and observations. Crucially, these assertions were intentionally designed to be inconsistent to serve as a benchmark for conflict resolution.
Comparing Methodologies
Quinlan compared three distinct architectures using this trial application:
- Prospector (Bayesian): This system required significant reformulation to handle the trial's complexity. It failed to detect the inherent inconsistency and produced a categorical interpretation that was logically invalid.
- INFERNO (Non-Bayesian): Developed previously by Quinlan, INFERNO uses probability bounds (lower bounds for truth and falsehood) rather than single points. It successfully flagged the inconsistency and suggested rectifications, but its categorical interpretation still contained anomalies.
- Ponderosa (Consistency-Based): The novel approach introduced in this paper. Ponderosa treats assertions as logical formulas and searches for maximally consistent subsets of evidence. It identifies "removal plans"—sets of evidence that must be discarded to resolve contradictions—and ranks them based on the validity measures of the excluded data.
Key Results and the Power of Consistency
Ponderosa identified six distinct ways to make the econometric model consistent. By removing the "least valuable" evidence (assertions A1 and A10), it produced a perfectly consistent set of categorical inferences. This highlights a shift in focus: rather than trying to smooth over contradictions with probabilities, Ponderosa identifies which evidence is most likely causing the conflict.
One of the most significant insights is the importance of non-directionality. Traditional systems often assume a hierarchical flow from evidence to conclusions. However, real reasoning is often bi-directional; if a conclusion is found to be false, we should be able to update our belief in the premises. Both INFERNO and Ponderosa allow for this "backwards" flow of information.
Conclusion
Quinlan suggests that the most robust framework for reasoning under uncertainty is a hybrid pipeline: use Ponderosa to identify a logically sound core of consistent data, and then apply a system like INFERNO to derive precise probability bounds from that subset. While Ponderosa is computationally more expensive than traditional methods, its ability to provide a logically valid foundation makes it a powerful tool for complex expert domains where evidence rarely adds up perfectly.
Summary updated