Abstract
This paper establishes a formal, axiomatic framework for updating preferences, bridging the gap between belief revision theory and preference modeling. While distance-based methods are intuitive, they often fail to produce a single, clear-cut preference order. To address this, we propose a "resolute" revision model that utilizes an agent's internal hierarchy of comparisons to resolve conflicts. Through the development of representation theorems, we demonstrate a one-to-one correspondence between operators satisfying specific logical postulates and mathematical constructions like faithful preference assignments.
Introduction
In fields ranging from economics to AI alignment, updating an agent's preferences in response to new, authoritative information is a critical task. For instance, an AI system might initially model a user's preference for Arthouse films over Biopics; however, if the user explicitly states a preference for Comedies over Arthouse, the system must "revise" its model. The challenge lies in performing this revision rationally while respecting structural requirements such as transitivity (if A > B and B > C, then A > C).
The Challenge of Multiplicity
The authors first explore Irresolute Revision, which follows classic propositional revision styles. This method uses distance metrics—specifically the Kendall tau distance, which counts the number of "flips" required to transform one ranking into another—to identify the new preference states closest to the original.
As shown in Figure 1, a significant drawback of this approach is that it often results in a set of possible preference orders rather than a single, actionable list. This "irresoluteness" is problematic for systems that require a definitive linear ranking to present to a user.
Resolute Revision and Faithful Assignments
To ensure a single, "resolute" linear order, the paper introduces the concept of a faithful preference assignment. This assumes that an agent possesses a "preference over comparisons"—a ranking of which parts of their current order they are most willing to sacrifice when a conflict arises.
The methodology is strictly axiomatic. The authors propose sets of postulates (logical requirements) and prove representation theorems. These theorems show that operators satisfying these axioms correspond exactly to specific mathematical constructions. For example, the resolute operator $\circ^f$ uses a faithful ranking of adjacent comparisons to iteratively build a new list, keeping the most "intense" links that do not contradict the new information, as conceptualized in the cycle-breaking structures of Figure 2.
Key Insights and Results
- The Mismatch Impossibility: Theorem 2 demonstrates that for most natural distance functions, it is mathematically impossible to guarantee a single-statement output while remaining "rational" by standard definitions. Achieving resoluteness requires moving beyond simple distance metrics.
- Axiomatic Alignment: The authors successfully adapted the Katsuno-Mendelzon (KM) postulates for the preference domain. They proved that their resolute operator satisfies a specific set of four "resolute postulates" ($R^r_1$ through $R^r_4$), providing a formal definition of rational resolute revision.
- Logic-Based Foundation: By situating preference revision within the broader family of belief change operators, the paper provides a rigorous foundation for AI researchers to build agents that can learn and adjust their understanding of human values over time.
Limitations and Future Work
The current model assumes the prior preference is a strict linear order. However, human preferences are often incomplete or partial. Future research directions include extending the resolute framework to handle partial orders and investigating complex distance functions that might avoid the "mismatch" problem without requiring a pre-existing ranking of comparisons.
Summary updated