Most early AI systems were reactive: they received an input and produced an output based on a fixed set of rules or a statistical probability. However, to operate in the "messy" real world—whether navigating medieval history as in Ghosts in the Machine: An AI Treatment of Medieval History or managing a modern supply chain—an agent needs more than a reflex. It needs a "mind" that maintains internal states, tracks long-term goals, and understands the motivations of others.
This research direction focuses on the Belief-Desire-Intention (BDI) framework and its modern evolution into LLM-based cognitive architectures. It moves AI from simple optimization toward "intentional agency," where behavior is driven by internal mental models rather than just external triggers.
The BDI Triad: Beliefs, Desires, and Intentions
The core intuition of this field, formalized by researchers like Philip R. Cohen and Hector J. Levesque in Speech Acts and Rationality, is that human-like reasoning can be decomposed into three functional layers:
- Beliefs: The agent’s internal representation of the world. This isn't just "data," but a subjective map that can be incomplete or even wrong.
- Desires: The set of all possible states the agent wants to achieve. These are often contradictory (e.g., "I want to save money" and "I want a new car").
- Intentions: The specific desires the agent has committed to acting upon.
As Adam Amos-Binks and R. Michael Young explain in Plan-Based Intention Revision, intentions provide stability. A rational agent doesn't drop its goal the moment a small obstacle appears; it revises its plan while maintaining its intention until that intention is either achieved or becomes impossible.
From Utility to Appropriateness
Traditional reinforcement learning often relies on utility maximization—mathematically picking the path with the highest reward. However, researchers like Alexander Vezhnevets at Google DeepMind argue in Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia that human behavior is often driven by a logic of appropriateness.
We act based on who we are (identity), what we remember (episodic memory), and what the social situation demands (norms). By separating cognitive functions into modular components—like the "Desire System" proposed by Sho Yoshida in On The System Of Concepts Relations And Outline Of The Natural Language System—researchers can build agents that behave more predictably and ethically in social settings.
Grounding LLMs in Mental States
The rise of Large Language Models (LLMs) has given this field a powerful new engine. While LLMs are naturally "stochastic," researchers are using BDI principles to ground them. In Can Large Language Model Agents Simulate Human Trust Behavior?, Chengxing Xie and colleagues demonstrate BDI Prompting.
Instead of asking an LLM to "decide," they prompt it to explicitly state its internal beliefs, desires, and intentions before outputting an action. This structured reasoning ensures the agent's behavior is aligned with a coherent internal state. This approach has been used to simulate everything from financial markets in TwinMarket to urban resident behavior in MobileCity.
Social Agency and Collective Intentionality
A major frontier in this research is moving from single agents to Human-Agent Teaming. Rick Evertsz and John Thangarajah argue in A Framework for Engineering Human/Agent Teaming Systems that for humans to trust AI, the AI must share a "collective intention."
This requires the agent to have a Theory of Mind (ToM)—the ability to infer the mental states of others. Benchmarks like NegotiationToM by Chunkit Chan and others test whether AI can track a partner's hidden preferences during a negotiation. Furthermore, Pietro Panzarasa and Nicholas R. Jennings describe this as Social Mental Shaping, where an agent's internal BDI state is dynamically updated by its social roles and the influence of its peers.
Why This Works: The Power of Abstraction
The reason the BDI-style cognitive architecture remains relevant from the 1970s to the LLM era is its power as a representational strategy. As Samir Chopra and Laurence White point out in Attribution of Knowledge to Artificial Agents and their Principals, treating a complex system as an "intentional agent" allows us to predict and regulate its behavior without needing to understand every line of code.
By modeling AI through the lens of beliefs and goals, we create a shared language between human intuition and machine logic, making autonomous systems more explainable, more social, and ultimately more capable of complex collaboration.