This paper, originally published by SRI International in 1987, addresses a fundamental "black box" problem in artificial intelligence: providing human-readable justifications for decisions made by systems that reason under uncertainty. While traditional logic-based systems can offer a "proof tree" to explain their conclusions, systems utilizing numeric measures—such as the Dempster-Shafer theory of belief functions—often struggle to translate complex mathematical weights into intuitive explanations.
The Challenge of Evidential Reasoning
By the late 1980s, expert systems were increasingly moving beyond simple "if-then" logic to handle real-world ambiguity using formalisms like Certainty Factors, Bayesian networks, and belief functions. However, these numeric approaches introduced opacity. When a system assigns a 0.75 support value to a hypothesis, users naturally ask, "Why?"
In a standard logic program, one can trace a single path of rules. In an evidential system, dozens of independent pieces of evidence might be "fused" together. There is no single path; instead, there is a complex web of interacting weights. The authors aimed to bridge this gap by creating a facility that translates these weights into meaningful justifications.
Methodology: The Gister System
The authors introduced an explanation facility for Gister, a system based on the Dempster-Shafer theory. Their approach relies on several key pillars:
1. The Analysis Graph as a Proof Tree
To provide structure to the reasoning process, Gister treats the flow of information—including operations like fusion, translation, and projection—as a data-flow graph. This graph serves as the structural equivalent of a logical proof tree, allowing the system to track how individual inputs contribute to the final conclusion.
2. Sensitivity Analysis via Discounting
To identify which piece of evidence is most influential, the system performs a "what-if" analysis. It uses a discounting operation to systematically reduce the credibility of a specific input (from 1.0 down to 0) and measures the resulting shift in the final belief interval $[Support, Plausibility]$. This allows the system to rank evidence by its impact on the outcome.
3. Sensitivity Space Plots and Information Theory
The paper introduces a 2D coordinate system to categorize evidence based on its effect on both Support and Plausibility. This distinguishes between evidence that builds a case versus evidence that merely reduces confusion. Additionally, the authors utilize Yager’s measures of Specificity (commitment to a precise outcome) and Consonance (internal agreement of evidence) to refine high-level explanations.
Case Study: The "Lost Dog" Scenario
The methodology was demonstrated using a simplified "Lost Dog" (Rufus) problem. A detective must locate a dog based on four conflicting pieces of evidence: the owner's last sighting, a police report from a neighboring county, a coworker's sighting on a highway, and historical data regarding the dog's habits.
The sensitivity analysis allowed the system to answer specific queries with ranked evidence:
- "Why do you believe Rufus is in Los Altos?" The system identified the coworker's sighting as the primary justification due to its high sensitivity ($\Delta Spt = 0.97$).
- "Is there any reason to believe he is NOT there?" The system pointed to historical data showing Rufus rarely stays in Palo Alto as a counter-productive factor.
- Filtering Noise: The analysis correctly ignored the owner's initial sighting because the time elapsed rendered its impact on the current location zero.
Insights and Limitations
A key insight of the paper is that the most "important" reason for a conclusion is not necessarily the one with the highest weight, but the one whose absence would change the conclusion the most. To make these findings accessible, the authors suggested using natural language templates to describe mathematical relations in real-world terms.
However, the current method is limited to univariate analysis, varying only one piece of evidence at a time. It does not yet account for synergistic effects where multiple weak pieces of evidence become powerful only when combined. Furthermore, the computational cost of re-running the analysis graph for every input could pose scalability challenges for massive, real-time systems.
Summary updated