On Emergent Social World Models -- Evidence for Functional Integration of Theory of Mind and Pragmatic Reasoning in Language Models

Polina Tsvilodub, Jan-Felix Klumpp, Amir Mohammadpour, Jennifer Hu, Michael Franke

TL;DR

Do large language models (LLMs) truly understand the social world through unified internal representations, or are they just memorizing disjointed conversational tricks? This paper applies cognitive neuroscience techniques to show that LLMs recruit shared, integrated computational mechanisms for both general Theory of Mind (ToM) and pragmatic language reasoning. By locating and ablating these "social subnetworks," the authors demonstrate that LLMs develop emergent, interconnected social world models rather than isolated competencies.

What Problem Does This Paper Address?

While LLMs show impressive capabilities, debate continues over whether they are merely "stochastic parrots" or possess genuine internal "world models." While world models have been successfully demonstrated in narrow, rule-bound environments (like Chess or Othello), it is less clear if they exist for complex, ambiguous human domains like social interaction.

Specifically, this paper investigates the relationship between Theory of Mind (ToM)—the ability to attribute unobservable mental states like beliefs and desires to others—and pragmatic reasoning (understanding context-dependent meaning, such as irony, hints, or sarcasm). The authors tackle a fundamental question: Do LLMs use separate, specialized circuits for these tasks (the functional specialization hypothesis), or do they share a unified "social world model" (the functional integration hypothesis), much like the overlapping neural networks found in the human brain?

Methods

The authors use a two-pronged approach combining behavioral and mechanistic analysis:

  1. Behavioral Evaluation: They test 48 different LLMs across dozens of tasks to see if ToM and pragmatic performance correlate and predict each other better than general language proficiency.
  2. Functional Localization & Ablation: Borrowing a technique from cognitive neuroscience, they identify specific "subnetworks" (groups of neurons) within the LLMs that are highly active during ToM tasks compared to closely matched control tasks (e.g., reading a story about a character's false belief vs. a story about a physically outdated photograph). They then perform causal ablation experiments—setting the activations of these localized neurons to zero—to observe the impact on the model's performance across different domains.

Datasets

The study leverages a massive evaluation suite consisting of:

  • Pragmatics Benchmarks: 16 datasets covering 10 phenomena (e.g., scalar implicatures, irony, sarcasm, indirect speech) drawn from benchmarks like Ma et al. (2025) and PUB.
  • Theory of Mind (ToM) Benchmarks: 22 datasets categorized under the ATOMS framework (which breaks ToM down into beliefs, intentions, desires, emotions, knowledge, percepts, and non-literal communication), including BigToM, OpenToM, and ToMi.
  • Linguistic Baselines: BLiMP (grammar) and SNLI (natural language inference).
  • Novel Localizer Suite: The authors introduced a large, synthetically generated dataset (via GPT-5) containing 1,400 stimuli across four suites (LatentBeliefs, CommunicativeIntent, GameBeliefs, MoralIntent) based on validated human fMRI experimental materials.

Key Insights and Ideas

  • Social Integration over Specialization: LLMs do not treat social reasoning and conversational pragmatics as completely separate tasks. Instead, they cross-purpose internal mechanisms, suggesting that efficient compression principles during pretraining favor the emergence of a unified social world model.
  • Causal Proof of Social Circuits: The most striking "aha" moment comes from the ablation studies. Damaging the specific neurons localized for ToM didn't just hurt ToM performance; it systematically degraded pragmatic reasoning as well, while leaving general grammatical and linguistic abilities largely intact.
  • Shared Tracking Mechanisms: Further exploration revealed that these social subnetworks are also heavily involved in tracking physical entities (e.g., box contents over a series of shifts), suggesting that social reasoning in LLMs builds directly upon foundational tracking mechanisms.

Key Results

  • Across 48 models, there is a strong, significant positive correlation ($r = 0.68$) between average accuracy on ToM and pragmatics tasks.
Figure 1: Average accuracy of 48 models on ToM (x- axis) and pragmatics (y-axis) tasks.
  • Statistical modeling showed that a model's ToM performance is a significantly better predictor of its pragmatic accuracy than its performance on general linguistic benchmarks (BLiMP and SNLI).
  • In the global ablation analysis, knocking out the localized ToM subnetworks caused a significant drop in both ToM and pragmatic performance, but had no credible negative effect on baseline linguistic benchmarks. This asymmetry strongly supports the functional integration hypothesis.
Figure 3: All: Plots show the mean causal effects (y-axis), i.e., difference in accuracy relative to the non-ablated baseline when ablating the critical (ToM) subnetworks or the control (least active) sub-networks (x-axis), for different test sets (colors and shapes). 95% CIs indicate change in by-dataset accuracy across localizer suites. Top row: Fictitious, stylized result that could come from our experiments (left-most plot), and a visualization of the three main predictions made by the functional integration hypothesis. Bottom row: Four observed outcomes at different levels of analysis. Results from the “global analysis” (marginalizing over all language models and localizers) are shown on the left; followed by example results for data from only one language model (averaging over localizers) and two localizer sets (averaging over language models). Predictions are supported for the “global analysis,” but not for all subunits at finer levels of analysis (e.g., the right-most example in the bottom row).

Limitations and Future Work

  • Coarse Interventions: Setting neuron activations to zero is a relatively blunt instrument. The authors note that future work should employ finer-grained mechanistic interpretability tools like activation patching or circuit discovery.
  • English and Cultural Bias: All datasets were exclusively in English. Pragmatics and social norms vary wildly across cultures, meaning these findings might not seamlessly generalize to other languages.
  • Lack of Human Validation for Stimuli: While the synthetic localizer suite was inspired by human fMRI studies, the newly generated text prompts have not been validated on actual human brains to confirm they activate the same biological pathways.
  • Data Contamination Risk: Because some benchmarks have been public for years, there is an inherent risk that newer models encountered them during pretraining, potentially inflating baseline scores.

Summary updated

Abstract

This paper investigates whether LMs recruit shared computational mechanisms for general Theory of Mind (ToM) and language-specific pragmatic reasoning in order to contribute to the general question of whether LMs may be said to have emergent "social world models", i.e., representations of mental states that are repurposed across tasks (the functional integration hypothesis). Using behavioral evaluations and causal-mechanistic experiments via functional localization methods inspired by cognitive neuroscience, we analyze LMs' performance across seven subcategories of ToM abilities (Beaudoin et al., 2020) on a substantially larger localizer dataset than used in prior like-minded work. Results from stringent hypothesis-driven statistical testing offer suggestive evidence for the functional integration hypothesis, indicating that LMs may develop interconnected "social world models" rather than isolated competencies. This work contributes novel ToM localizer data, methodological refinements to functional localization techniques, and empirical insights into the emergence of social cognition in artificial systems.
BibTeX
@inproceedings{tsvilodub2026emergent,
  author = {Polina Tsvilodub and Jan-Felix Klumpp and Amir Mohammadpour and Jennifer Hu and Michael Franke},
  title = {{On Emergent Social World Models -- Evidence for Functional Integration of Theory of Mind and Pragmatic Reasoning in Language Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1735},
}
Figure