The Generation of Explanations within Evidential Reasoning Systems

TL;DR

This paper introduces a novel method for generating explanations in evidential reasoning systems, which handle uncertainty using belief functions. By employing sensitivity analysis on the system's analysis graph, it allows users to understand why a conclusion was reached, addressing a key limitation of existing uncertain reasoning systems. This work bridges the gap between complex probabilistic reasoning and user-understandable justifications.

What Problem Does This Paper Address?

Expert systems are lauded for their ability to explain their reasoning, fostering user trust and aiding in knowledge engineering. However, a significant challenge arises when these systems must reason under uncertainty, particularly those using numeric measures like Dempster-Shafer belief functions. While these systems excel at representing partial ignorance and degrees of precision, they traditionally lack a robust mechanism for explaining their conclusions. Existing methods for generating explanations in simpler, deterministic systems (like logic programs) rely on tracing proof trees, which become unwieldy and incomprehensible when dealing with the complex interplay of uncertain evidence. This paper tackles the problem of providing meaningful, transparent explanations for systems that employ evidential reasoning, a powerful framework for handling uncertainty.

Methods

The core of the paper's contribution lies in augmenting an evidential reasoning system with a versatile explanation facility based on sensitivity analysis. The authors leverage the analysis graph, an internal representation of how evidence flows and combines within the system, as the foundation for their explanation generation.

Key methodological steps include:

  • Discounting Nodes: Introducing "discounting nodes" into the analysis graph. These nodes represent the credibility of primitive evidence sources, allowing for systematic variation of their influence.
  • Sensitivity Computation: Calculating the sensitivity of a conclusion (e.g., the support or plausibility of a hypothesis) with respect to the credibility of each primitive evidence source. This is done by computing partial derivatives of the conclusion's metrics with respect to the credibility parameter.
  • Sensitivity Space Plotting: Visualizing these sensitivities in a "sensitivity space" (plotting support vs. plausibility sensitivities). This helps identify which pieces of evidence strongly support, argue against, or are neutral to a conclusion.
  • Characterizing Bodies of Evidence (BOEs): Extending sensitivity analysis to entire bodies of evidence by analyzing their "specificity" (degree of commitment to precise propositions) and "consonance" (a measure related to entropy and uncertainty). This allows for higher-level explanations about the overall impact of evidence.

This approach moves beyond simply recapitulating steps; it quantifies the impact of each piece of evidence on the final conclusion.

Datasets

Not discussed in the paper. The paper uses a hypothetical example of finding a lost dog ("Rufus") to illustrate its methods.

Key Insights and Ideas

  • Analysis Graph as Explanation Foundation: The internal analysis graph, which models the flow of evidence and operations (fusion, translation, projection), is a natural analog to proof trees in logic and serves as an excellent data structure for explanation generation.
  • Credibility as the Key Input for Sensitivity: Instead of varying probabilities directly (which is computationally infeasible for belief functions), the authors focus on varying the credibility of primitive evidence sources. This is a more manageable and intuitive approach.
  • Sensitivity Analysis Reveals True Impact: Sensitivity analysis effectively reveals which pieces of evidence were truly influential in reaching a conclusion, distinguishing between evidence that strongly supports, weakly supports, or even detracts from a hypothesis.
  • Beyond Single Hypotheses: The framework extends to analyzing entire bodies of evidence, allowing for explanations about the overall precision and consonance of the evidence contributing to a conclusion.
  • Bridging the Gap: The method provides a way to translate the complex, numeric outputs of evidential reasoning into understandable justifications, addressing the "black box" problem often associated with uncertain AI systems.

Key Results

The paper demonstrates its methodology through a detailed example of locating a lost dog. The sensitivity analysis reveals that:

  • The fact that the dog was at home at 8:00 AM had no bearing on the conclusion about its current location.
  • The police report, the coworker's sighting, and historical data were all necessary for the supporting argument.
  • The coworker's report was identified as the most important piece of evidence for the conclusion that the dog was in Los Altos.
  • The historical data (Rufus running away before) was found to have a negative consonance, suggesting it argued against the consensus, possibly due to the dog being found in Palo Alto once before.

Qualitatively, the paper shows how this approach can answer questions like "Why do you believe X?" or "Is there any reason to believe Y is not true?" by identifying the most influential pieces of evidence.

Limitations and Future Work

  • Computational Complexity: While the method is more tractable than directly varying belief functions, computing joint effects of multiple evidences numerically can still involve exploring a combinatorially large parameter space. The paper notes that whether such multivariate sensitivity analysis is useful for real problems remains to be determined.
  • Subjectivity of Assessments: The paper acknowledges that the initial assessment of evidence (e.g., credibility of reports) is subjective, a common challenge in all uncertain reasoning systems.
  • Future Work: The authors suggest that sensitivity information could be valuable for decision analysis, helping to determine if sufficient information is available or if more should be sought. They also propose exploring how to hypothesize information from particular sources to see its potential impact on a pending decision.

Summary updated

BibTeX
@inproceedings{1987generation,
  title = {{The Generation of Explanations within Evidential Reasoning Systems}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1987},
}
Figure