The Emergence of Ordered Belief from Initial Ignorance

Paul SnowAAAI 1994

Introduction

In classical Bayesian inference, the process of updating beliefs upon the arrival of new evidence requires a pre-existing "prior" probability distribution. This requirement creates a fundamental challenge: how can a system represent a state of total ignorance? The Principle of Indifference, which assigns equal probabilities to all possibilities, often fails because it imposes a specific numerical ordering that may be arbitrary or shift inconsistently when categories are refined.

This paper addresses the "unwillingness to order" propositions by their belief-worthiness when evidence is absent. The proposed framework allows evidence to speak for itself, establishing a partial ordering (e.g., "Proposition A is at least as likely as Proposition B") only when the evidence is sufficiently strong to warrant such a claim, thereby avoiding the forced creation of a complete numerical distribution from the start.

Axiomatic Framework

The methodology utilizes an axiomatic approach within a partitioned domain consisting of "atoms" (mutually exclusive and exhaustive possibilities) and "disjunctions" (combinations of these atoms). The framework is built upon six formal assumptions (A1-A6), with three being particularly critical:

  • Impartiality and Consistency: Logical implications must be respected; if proposition $S$ implies $S'$, then $S'$ must be at least as belief-worthy as $S$.
  • Recovery from Ignorance: This principle posits that if the likelihood of evidence given atom $s$ is significantly higher than the likelihood given atom $t$, a believer is finally justified in asserting that $s$ is more belief-worthy than $t$.
  • Quasi-additivity: This ensures that the ordering remains predictable when common elements are added to or removed from the propositions being compared.

From these axioms, an inference rule for simple disjunctions is derived: to compare two sets of possibilities, one only needs to evaluate the "best-supported" atom (the one with the highest likelihood) in each set.

The Surrogate Algorithm

While the framework prioritizes maintaining a partial ordering, practical decision-making often requires a single numerical probability distribution. The paper introduces a "surrogate" algorithm to map qualitative orderings into a usable distribution. This process operates in linear time ($O(N)$) relative to the number of atoms, making it computationally efficient.

Data structures for the surrogate probability distribution algorithm.

The algorithm functions by maximizing a constant $c$, which serves as a lower bound for the least-favored atoms. This approach minimizes the "epistemological burden" on the believer by ensuring that no more order is imposed than the evidence strictly requires. As shown in the data structures above, the algorithm tracks weights and running sums to maintain consistency with the derived order constraints.

Key Insights and Results

The theoretical analysis yields several significant results:

  1. The "Best-Supported" Shortcut: The discovery that comparing complex disjunctions depends only on the highest likelihood disjunct in each set significantly simplifies query handling in large domains.
  2. Qualitative Probability: The author proves that the proposed ordering satisfies the requirements of a partial qualitative probability, bridging the gap between pure logic and numerical probability theory.
  3. Two-Tiered Belief Model: The system aligns with the "Transferable Belief Model," maintaining a flexible partial ordering (the credal phase) and only collapsing it into a single distribution (the pignistic phase) when a decision or bet is required.

Limitations and Future Work

Despite its strengths, the model introduces subjective parameters, specifically the function $f$ and threshold $q$ used to determine when evidence is "strong enough" to warrant an ordering. Furthermore, because the system produces a partial order, it may return an "I don't know" response when evidence is insufficient to rank two propositions. Future research may explore the scaling of this model to convex sets of distributions to provide a more comprehensive range of possible probabilities.

Summary updated

BibTeX
@inproceedings{snow1994emergence,
  author = {Paul Snow},
  title = {{The Emergence of Ordered Belief from Initial Ignorance}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1994},
}
Figure