For an autonomous agent to be considered "rational," it is not enough for it to simply possess a list of goals and a database of facts. A truly rational agent must maintain a coherent internal state where its beliefs, desires, and intentions do not contradict one another. If an agent intends to achieve a goal it believes is impossible, or if it pursues two objectives that mutually sabotage each other, its behavior ceases to be purposeful.
Research into rational agency focuses on formalizing these requirements through logical constraints. By treating mental states as formal structures, researchers can define exactly what it means for an agent to be consistent and how it should update its plans as the world changes.
The Asymmetry of Beliefs and Intentions
A fundamental challenge in modeling agents is the "side-effect problem." If an agent believes that going to the dentist will inevitably cause pain, does that mean the agent intends to feel pain? In a naive logic, anything the agent believes will happen as a result of its actions might be incorrectly classified as a goal.
To solve this, Su et al. and the authors of Asymmetry Thesis and Side-Effect Problems propose the asymmetry thesis. This principle suggests that while an agent’s intentions should be consistent with its beliefs, they are not identical to them. Specifically, the authors of Asymmetry Thesis and Side-Effect Problems introduce "weak-realism," a semantic constraint where the set of worlds the agent considers possible merely overlaps with the worlds where its goals are realized. This prevents the agent from adopting every inevitable side effect as a personal objective.
Intention as a Filter for Deliberation
In the influential framework proposed by Cohen et al., intention is defined as more than just a desire; it is a persistent commitment. Once an agent adopts an intention, that commitment acts as a "filter" for all future reasoning.
This mechanism provides two types of consistency:
- Internal Consistency: The agent cannot adopt a new goal that contradicts its existing commitments.
- Means-End Consistency: As Balkanski formalizes, if an agent intends to perform a complex action, it must also intend the necessary sub-actions and believe there is a causal link between them.
By treating active intentions as fixed constraints, the agent avoids the computational cost of constantly re-evaluating every possible path, focusing only on those that align with its established plans.
Dynamic Maintenance and Capability Alignment
Rationality also requires an agent to be responsive to new information. If an agent learns that a goal has already been achieved or has become impossible, it must drop that goal to remain rational. Alechina et al. identify specific logical axioms that enforce this, ensuring agents do not waste resources pursuing satisfied objectives.
However, dropping a goal shouldn't always be permanent. Shapiro et al. argue that agents should be able to "readopt" goals if they previously believed they were impossible but later receive evidence to the contrary. This creates a dynamic relationship where goals are temporarily suspended but not forgotten, maintaining long-term coherence.
Furthermore, an agent's intentions must be grounded in its capabilities. The authors of Agent Capabilities: Extending BDI Theory introduce compatibility axioms ensuring that an agent only forms intentions for actions it believes it has the power to execute. This prevents the "irrational commitment" of an agent trying to perform tasks beyond its physical or computational reach.
Integrating Norms and External Constraints
In multi-agent or social environments, personal goals often conflict with external rules, or norms. Rationality in these contexts involves balancing internal desires with societal obligations. Shams et al. and Padget et al. demonstrate that this balancing act can be modeled using Answer Set Programming (ASP). By mapping agent plans to logical answer sets, they ensure that the chosen course of action is not only optimal but also logically defensible against conflicting norms.
Similarly, Endriss et al. show that communication protocols can be integrated as internal integrity constraints. This allows the agent's own reasoning engine to filter out illegal moves, ensuring that its behavior remains "conformant" to external rules without needing constant external monitoring.
Verification of Coherence
These formal logical structures allow for the automated verification of an agent's "sanity." Morgenstern applies this to character analysis, using first-order logic to determine if an agent's actions are "coherent"—meaning they are grounded in the agent's history and arise logically from its primary objectives. In a business context, van Zee uses these constraints to build "coherence checkers" that identify contradictions between a decision-maker's intended strategy and their beliefs about the market.
Ultimately, these constraints transform an agent from a simple reactive system into a rational actor capable of maintaining a stable, justifiable, and effective path toward its goals.
Key Foundational Work
Intention = Choice + Commitment (1987) — contributed 2 concept elements to this direction.
Asymmetry Thesis and Side-Effect Problems in Linear-Time and Branching-Time Intention Logics (1991) — contributed 1 concept elements to this direction.