Abstract
Large Language Models (LLMs) often struggle with complex Theory of Mind (ToM) tasks involving nested beliefs and naturalistic dialogue. We present Decompose-ToM, a training-free inference algorithm that recursively decomposes complex ToM questions into simpler perspective-taking subtasks. By simulating character knowledge based on physical presence and "knowledge-access" principles, Decompose-ToM significantly outperforms existing methods on higher-order and conversational ToM benchmarks, demonstrating particular robustness in long-context scenarios.
Introduction
Theory of Mind (ToM)—the ability to attribute mental states, beliefs, and knowledge to others—is a cornerstone of human social intelligence. While frontier LLMs perform well on basic ToM benchmarks like the Sally-Anne test, they frequently fail on higher-order nested beliefs (e.g., "Alice thinks that Bob thinks...") and in naturalistic settings where multiple agents track information asynchronously. Standard prompting techniques, such as Chain-of-Thought (CoT), often fail because models struggle to maintain multiple conflicting realities within a single context window.
The Decompose-ToM Framework
Decompose-ToM is an inference-time algorithm inspired by cognitive psychology, specifically "Simulation Theory" and the concept of "knowledge-access." Instead of solving a complex ToM query directly, the algorithm decomposes it into recursive sub-problems:
- Initialization and Reframing: The system identifies the target agent and reframes the question. For a query like "Where does Alice think Bob thinks the object is?", the system first simulates Alice's perspective and reframes the question to "Where does Bob think the object is?"
- Simulation and World State Updating: The algorithm processes the story statement-by-statement. For each event, it prompts the LLM to determine if the current agent (e.g., Alice) is aware of it based on their location. This creates a filtered "sub-story" containing only the facts known to that agent.
- Recursive Question-Answering: This process repeats for nested beliefs until the question becomes purely factual. The final answer is derived using standard CoT on the highly filtered, agent-specific text.
This approach ensures the LLM's context window acts as a literal representation of a character's mind, preventing interference from information the character shouldn't know.
Experimental Setup
The method was evaluated on two challenging benchmarks:
- Hi-ToM: Tests higher-order ToM (up to 4th order) involving item movement and deceptive characters.
- FANToM: A naturalistic, dialogue-based dataset requiring models to track information across multiple speakers entering and leaving conversations. Evaluations included both short and long-context variants.
Results and Analysis
Decompose-ToM demonstrated substantial performance improvements across all tested models, including GPT-4o, Llama-3-70B, and Gemini Flash.
Key findings include:
- Significant Accuracy Gains: On the Hi-ToM dataset, GPT-4o achieved a 28.13% average accuracy jump over the SimToM baseline. Llama-3-70B saw improvements as high as 44.2% in 4th-order tasks.
- Inverse Scaling with Difficulty: Unlike baseline methods that collapse as complexity increases, larger models using Decompose-ToM actually improved performance from 1st-order to 4th-order tasks.
- Robustness to Context Length: On the FANToM dataset, Decompose-ToM reduced the performance gap between short and long stories to just 0.9%, compared to a ~7.5% drop for baselines. This suggests that perspective-based filtering serves as an effective dynamic context-compression technique.
Discussion and Limitations
The success of Decompose-ToM highlights the value of grounding AI inference in developmental psychology. By physically rewriting the prompt to match an agent's knowledge, the model avoids the cognitive load of juggling multiple realities.
However, the method has notable limitations. It assumes a strictly chronological narrative and cannot easily handle non-linear storytelling. Additionally, the statement-by-statement processing is computationally expensive. Finally, the algorithm exhibits "snowballing errors" in smaller models; if a model like Llama-3-8B incorrectly judges an agent's knowledge early in the simulation, the error propagates through the recursive steps, leading to performance degradation on higher-order tasks.
Future work will focus on improving computational efficiency and integrating this recursive reasoning into autonomous agent frameworks.
Summary updated