Abstract
This paper introduces analytical methods to evaluate the impact of "fuzzy" expert-provided certainty factors on the conclusions of rule-based systems. By employing sensitivity functions and second-order probabilities, the author provides a framework to measure the robustness of inference networks, demonstrating that the shape of uncertainty distributions is mathematically determined by the system's underlying combining functions.
Introduction
Knowledge-based (KB) systems often rely on certainty factors—numerical values representing trust in data or rules. However, these values are frequently subjective estimates. A critical question arises: how do small errors in these inputs affect the final output? Early systems like MYCIN and Prospector lacked formal tools to measure this sensitivity, risking "brittleness" where minor input variations lead to drastically different conclusions. This paper formalizes uncertainty analysis to ensure system robustness through a formal mathematical toolkit.
Analytical Framework
The author proposes two complementary approaches to move beyond simple-minded interpretations of certainty factors:
- Sensitivity Analysis: Defined as the partial derivative of the combining function (the math used to merge evidence), this measure tracks how input changes propagate through a network. The paper extends this to "path sensitivity" for complex Directed Acyclic Graphs (DAGs), providing a recursive algorithm to calculate how sensitive a final hypothesis is to any leaf node.
- Second Order Probabilities: Instead of fixed points, certainty factors are treated as distributions (the "uncertainty of the uncertainty"). Using the algebraic theory of uncertainty and the Central Limit Theorem, the author approximates the probability distribution of global certainty factors, proving that the "shape" of the final uncertainty is dictated by the system's math, not just the input data.
The Law of Average Sensitivity
A key theoretical contribution is the "Average Sensitivity Law" (Proposition 6). It states that for any valid combining function over the interval (-1, 1), the average sensitivity is exactly 1. This implies a fundamental trade-off: a function that is exceptionally stable in one region must necessarily be highly sensitive in another to maintain this average. A good system must balance being stable enough to avoid noise while remaining sensitive enough to weigh evidence effectively.
Comparative Analysis of Expert Systems
The paper compares the sensitivity profiles of the Emycin and Prospector systems. While both are designed for robustness, they exhibit different behaviors as inputs approach extreme values.
As seen in Figure 6, the sensitivity functions for Emycin and Prospector are relatively smooth, whereas the counter-example (c) shows extreme spikes (singularities). This indicates a lack of robustness despite the fact that the counter-example satisfies standard algebraic axioms, proving that axioms alone are insufficient for system stability.
Robustness and Combining Functions
The choice of combining function $g(x,y)$ is the primary determinant of system health. The author uses synthetic counter-examples to illustrate how a function can meet standard criteria but fail in practice due to non-linearity and singularity points.
Figure 2 illustrates the "bad" combining function used in the counter-example. The sharp, non-differentiable transitions in the mapping of $x$ and $y$ to the combined certainty factor are the root cause of the instability observed in the sensitivity analysis. This visualizes why certain mathematical formulations lead to brittle AI behavior.
Conclusion
The paper concludes that while compositional methods are computationally efficient, they require rigorous "off-line" sensitivity analysis before deployment. While the addition of second-order probability calculations adds overhead, these tools are essential for stress-testing a knowledge base. Future work involves automating these tests to ensure that the mathematical "shape" of uncertainty remains within acceptable bounds for real-world applications.
Summary updated