TL;DR
This paper argues that "reasoned assumptions" in artificial intelligence, often viewed through a logical or epistemological lens, are better understood as economic preferences or desires of an agent about its own mental state. This reframing connects AI reasoning to decision theory and economics, suggesting that AI systems can be seen as economic agents making choices to optimize their internal states.
What Problem Does This Paper Address?
Artificial intelligence systems often need to make decisions and draw conclusions with incomplete information. This is typically handled using "non-monotonic" reasoning, where new information can invalidate previous conclusions. The paper addresses how to formally model and understand these reasoning processes. Traditionally, these have been framed as issues of belief, self-knowledge, and consistency. However, the author proposes that this logical/epistemological view misses a crucial aspect: the agent's internal "preferences" or "desires" about what its own beliefs should be. The paper aims to bridge the gap between AI reasoning formalisms and established theories in economics and decision theory.
Methods
The paper introduces the concept of "simple reasons" as a formal tool. A simple reason is expressed as A || B -> C, meaning "if all of A is believed and none of B is believed, then C should be believed." Conclusions drawn from these reasons are called "reasoned assumptions." The core of the method lies in interpreting these simple reasons not as statements of fact, but as preferences of the agent.
The paper then defines "admissible states" of an agent's mental state (e.g., beliefs) as those that satisfy all of its component simple reasons. This is analogous to how an economic agent's preferences define a set of desirable outcomes. The paper also discusses "groundedness" (local and strict) as a way to ensure that these admissible states are not arbitrary but are derived from the initial set of beliefs and reasons. This leads to the idea of "expansions" or "extensions" of an initial state, which are admissible supersets.
Datasets
Not discussed in the paper.
Key Insights and Ideas
- Reasoned Assumptions as Preferences: The most significant insight is reframing reasoned assumptions from epistemological statements to economic preferences. This means an AI agent isn't just trying to know what's true, but is actively "choosing" or "desiring" certain configurations of its own beliefs.
- Self-Choice over Self-Knowledge: The fundamental relation in this framework is not "self-knowledge" but "self-choice" or "self-determination." The agent is making decisions about its own mental makeup.
- Economic Foundations: The justification for these reasoning structures comes from decision theory and economics, not logic or epistemology. This suggests that AI reasoning can be understood through the lens of economic agents.
- Group Decision Making Analogy: The paper draws a strong parallel between reasoned assumptions and group decision-making. Just as economists model how individual preferences combine to form group preferences, AI agents can be seen as combining their internal "preferences" (reasons) to form a coherent mental state.
- Pareto Optimality Connection: The paper demonstrates that Pareto optimal outcomes in group decision-making problems (where no individual can be made better off without making another worse off) correspond to admissible states in the reasoned assumption framework. This highlights the economic nature of these AI concepts.
Key Results
- The paper establishes that Pareto optimal sets of invitations in a party invitation scenario (where guests have preferences) are exactly the admissible states defined by reasoned assumptions.
- It shows that strictly grounded expansions of a set of reasons correspond to fixed points of a "closure" operator, which can be interpreted as equilibria in an economic sense.
- The paper suggests that the "economic man" idealization, while perhaps not a perfect model of humans, can be a useful building block for understanding the elementary components of AI systems.
Limitations and Future Work
- Abstraction from Human Complexity: The paper acknowledges that its interpretation takes liberties with standard economic theory, particularly by allowing group membership to vary with choice and by considering "preferences about preferences." It notes that this leads to an interesting extension of usual economic theory.
- Non-Monotonicity and Economics: The paper points out that non-monotonic reasoning rules, which are pervasive in AI, violate many standard economic axioms (consistency, transitivity, independence of irrelevant alternatives). This suggests a need for new idealized psychologies for economics that are closer to human computational limitations.
- Ambiguity and Subjective Probability: The paper notes that because AI agents are non-atomic and decisions can be group decisions, outcomes may be ambiguous (multiple incomparable outcomes). It suggests that developing a theory of subjective probability by measuring these ambiguities could be a fruitful direction, fitting AI's practical needs better than foundational roles.
- Future Work: The author is studying "rational psychology" and preparing a comprehensive treatment of these ideas.
Summary updated