TL;DR
This paper proposes a formal, axiomatic framework for understanding "belief updates" – how our beliefs change when new evidence arrives. By drawing parallels to the foundational axioms of probability, the authors show that belief updates in a probabilistic setting are fundamentally tied to likelihood ratios, offering a principled way to reason about belief revision.
What Problem Does This Paper Address?
In artificial intelligence and decision-making, a crucial aspect is how to update our beliefs when presented with new information. While methods for representing and manipulating "absolute belief" (e.g., the probability of a statement being true) are well-established, the process of quantifying changes in belief – belief updates – has been less formally grounded. Existing methods like the MYCIN certainty factor model or Dempster's Rule, while practical, lacked a rigorous theoretical foundation. This paper aims to provide such a foundation by developing a set of axioms for belief updates, analogous to Cox's foundational work on probability axioms.
Methods
The paper's core contribution is an axiomatic approach to belief updates. The authors first review Cox's axioms for absolute belief (clarity, completeness, scalar continuity, context dependency, consistency, hypothetical conditioning, and complementarity), which led to the conclusion that probability is the only admissible measure of absolute belief.
They then propose analogous axioms for belief updates:
- Definition of Belief Update: A belief update, $U(H,E,e)$, combined with a prior belief $H|e$, determines the posterior belief $H|Ee$ via a function $f$: $H|Ee = f(U(H,E,e), H|e)$. This function $f$ is assumed to be continuous and monotonically increasing.
- Consistency Property for Belief Updates: If the arguments of a belief update are logically equivalent, the updates must be equal.
- Combination Property for Belief Updates: Updates from sequential pieces of evidence can be combined directly. This leads to a functional equation that, when solved, implies that the transformation of belief updates must be additive.
From these axioms, the authors derive that any belief update must be a monotonic transformation of the difference between the posterior and prior belief. In a probabilistic context, this leads to the key result that belief updates must be monotonic transformations of the likelihood ratio, $x(H,E,e) = p(E|H,e) / p(E|\neg H,e)$.
Datasets
Not discussed in the paper.
Key Insights and Ideas
- Bridging the Gap: The paper successfully bridges the gap between intuitive notions of belief change and formal mathematical principles, much like Cox did for probability.
- Likelihood Ratio as the Core: The most significant insight is that, under a reasonable set of axioms, belief updates in a probabilistic setting are fundamentally linked to likelihood ratios. This provides a principled justification for why likelihood ratios are so central to many reasoning systems.
- Functional Equations as a Tool: The use of functional equations (derived from the combination property) is a powerful technique for characterizing the structure of belief updates.
- Modularity and Independence: The paper explores the concept of "modularity" in belief updates, showing how it relates to conditional independence assumptions in probabilistic reasoning. This offers a formal way to understand when the impact of one piece of evidence on a belief is independent of another piece of evidence, given the hypothesis.
Key Results
- The paper formally demonstrates that any measure satisfying the proposed axioms for belief updates must be a monotonic transformation of the difference between posterior and prior beliefs.
- In a probabilistic context, this implies that belief updates must be monotonic transformations of the likelihood ratio $x(H,E,e)$.
- The paper derives the "modularity property" for belief updates, showing that under certain independence assumptions, the update for a hypothesis with evidence $E_2$ given prior evidence $E_1$ is the same as the update with $E_2$ alone.
- It is shown that the likelihood ratio itself satisfies the derived axioms and properties, making it a legitimate belief update.
Limitations and Future Work
- Focus on Belief Entailment: The axiomization primarily focuses on the process of "belief entailment" (deriving beliefs from assessed beliefs), rather than problem formulation or belief assignment.
- Intuitive Justification: While the axioms are presented as capturing intuitive properties, the paper acknowledges that the primary goal is to stimulate discussion and critical examination of these properties, rather than definitively proving they are the only correct ones.
- Future Work: The authors hope this formal explication will facilitate critical discussion, useful extensions of the approach, and better judgment of the method's appropriateness for specific domains. They suggest that the axioms can serve as a tool for identifying dissatisfaction with the update approach and for promoting consistent use of methodologies that manipulate belief changes.
Summary updated