Language models asked "where does Sally think the ball is?" tend to answer with where the ball actually is. That is the classic failure in theory of mind, the ability to track what other agents believe, including beliefs that are wrong. In graph learning there is a close analogue. A graph neural network forms each node's representation by averaging in its neighbors' information. Where connected nodes do not share labels, that averaging overwrites what is distinctive about the node, and the fixes that work learn, per node, how much to trust the neighborhood versus the node's own features.
This note proposes treating belief tracking the same way. An agent's belief at each step mixes two sources: the updated state of the world, and the agent's own previous belief carried forward. The weight on the world should equal the probability that the agent observed the change. The gate is not a black box. It is a soft, learned version of the "seeing" operator that epistemic logicians use to model who observes what. Nested beliefs ("Anne thinks Bob thinks…") become products of such gates along a chain of agents. The practical bet is narrow and testable. When stories state plainly who witnessed what, a parser plus a symbolic tracker should already solve them. When witnessing must be inferred from indirect cues such as rooms, doors, glances, or partial sight, a calibrated soft gate should beat both hard symbolic tracking and fine-tuned language models. The advantage should grow with nesting depth.
Core Claim
The two research lines meet at one equation. Let $\sigma=(a_1,\dots,a_m)$ be a chain of agents, so $\sigma=(\text{Anne},\text{Bob})$ means "what Anne believes Bob believes." Let $b^\sigma_t$ be that chain's belief, a distribution over values of some state variable such as an object's location, after event $t$. Let $w_t$ be the true world state after the event. The proposal models
$$b^\sigma_t = g^\sigma_t\, w_t + (1-g^\sigma_t)\, b^\sigma_{t-1},$$
where $g^\sigma_t\in[0,1]$ is the estimated probability that every agent in the chain observed event $t$ and that each agent saw the next one in the chain observing it.
This is structurally the per-node balance parameter of CPF, which arbitrates between label propagation along edges and a node's own feature prediction. It is also the adaptive initial residual of AIR, which keeps a node's original signal from being washed out by repeated aggregation. Here the "residual" is the persisted prior belief, and washing it out is exactly the reality bias.
The analogy carries one substantive consequence. Optimality of Message-Passing Architectures for Sparse Graphs proves that the Bayes-optimal node classifier interpolates between feature-only and neighborhood-aggregating computation according to signal strength. The analogous statement here is elementary but important. If observation is a latent binary event with posterior probability $\pi$ given textual or spatial cues, the belief that minimizes expected log loss is the $\pi$-weighted mixture above. A symbolic tracker implicitly rounds $\pi$ to 0 or 1. It is therefore exact when cues are unambiguous and miscalibrated otherwise, and for a chain of length $m$ its errors compound across $m$ gates.
What is new is not a model that tracks beliefs; symbolic trackers exist. It is the claim that the observation gate is the right locus of learning, and that making it soft and cue-conditioned is what matters under ambiguous observability.
Relation to Prior Work
The epistemic side motivates the proposal and supplies its semantics. A Logical Analysis of Hanabi replaces "knowing that" with an observability operator and shows that the game's knowledge dynamics fit in modal depth 2. We borrow that operator as the thing the gate estimates, but make it probabilistic and learned rather than given. Reasoning about only Knowing with Many Agents argues for a $K45$-style system without the truth axiom for nested beliefs. Our update rule respects this: $b^\sigma$ is never forced toward $w$ unless the gate opens, so nested false beliefs about ignorance are representable.
Two benchmark papers supply the evidence for the gap. ExploreToM generates adversarial stories from a program with asymmetric belief updates and finds that failures in physical state tracking propagate into mental-state failures. EnactToM shows a "saying–doing" gap in embodied coordination. There, observability is determined by walls and radio range rather than stated in text, which is precisely the regime where we expect a soft gate to matter.
The graph side supplies the method and constrains it. SimP-GCN and GCN with node feature kernels show that learning how much to trust structure versus features is what rescues graph networks when neighbors disagree. The kernel paper also assumes features are clean while structure is noisy. We face the reverse risk, since the cue features that drive the gate may themselves be misleading. That is why the experiments include adversarial cues. DIGNN separates the two channels explicitly for fraud detection. We keep the channels separate by construction: world state versus persisted belief. None of these graph papers involve agents or nested beliefs, and none of the ToM papers learn where the update rule's gate sits.
Research Program
The system has three parts.
First, a language model reads a story or an embodied episode log. It emits a sequence of events, each with the state variable it changes, plus a set of observability cues for each agent–event pair. Cues include co-location, explicit mentions of looking or leaving, and door and wall relations in EnactToM scenes.
Second, a small gate network maps these cues to $g^{a}_t$ for single agents and to second-order terms "agent $a$ saw agent $b$ observe." It composes them into $g^\sigma_t$ for chains up to depth 4. The composition starts as a product, which is exact under conditional independence. A learned correction handles cases such as "they watched together," where the terms are correlated.
Third, the update rule above runs forward in time, and a question is answered by reading out $b^\sigma_T$.
The gate network is trained on ExploreToM-generated stories with ground-truth observability taken from the generator's program. The rest of the pipeline is fixed and interpretable, so every error can be attributed either to the parse or to a specific gate.
The central data contribution is an implicit-observability variant of ExploreToM. We keep each story's underlying program and ground-truth beliefs, but rewrite the witness statements so that observation must be inferred. Examples: "Anne stepped into the hallway just as…", or a character whose line of sight is described only spatially. The ambiguity is graded, with a controlled fraction of cues genuinely uninformative, so that the calibrated $\pi$ is known.
For the embodied extension, the same gates drive communication. An agent sends a message about variable $x$ to partner $b$ when the predicted discrepancy between $b^{(b)}$ and $w$ is large and the fact matters for $b$'s task. Messages are ranked by that discrepancy under the radio budget. This targets the two EnactToM failure modes, withholding critical information and misallocating messages, directly.
Baselines
Reality-default reference. The naive reference point always answers with the true world state, which is the gate fixed at 1. The "always persist" gate fixed at 0 is its mirror. Together they bracket the problem and quantify how much of each split is false-belief questions.
Prompted frontier models. We run chain-of-thought prompting on the same models used in the benchmark papers, with a perspective-taking prompt tuned on the development split. ExploreToM reports that on certain complex action sets GPT-4o scored 9% and Llama-3.1-70B-Instruct scored 0% (ExploreToM). This baseline shows how far unaided models are from the problem, but it is not the one to beat.
Fine-tuned model. The strongest learned competitor is Llama-3.1-8B fine-tuned on the 79,700 ExploreToM triples. It gained +27 points on ToMi and +29 on Hi-ToM (ExploreToM). The file reports only gains, not absolute accuracies, so we will reproduce it. For fairness, our gate network and the language-model parser receive the same training stories and the same number of gradient steps as this baseline. The parser is also an 8B model, fine-tuned on the same data.
Symbolic tracker. The strongest structured competitor is a symbolic belief tracker, essentially the ExploreToM generator's own update semantics, run on the same parser's output. Observation is decided by a hard threshold tuned on development data. The file has no reported result on Lacuna for this baseline, so it is built here.
Ablations isolating the gate. Three ablations target the key ingredient. The first replaces the learned gate with the hard threshold while keeping everything else identical. The second removes second-order terms, so that $g^{(a,b)}=g^a g^b$. The third keeps the gate but conditions it only on event text, dropping spatial cues.
What counts as winning. Beating the baselines means two things on the implicit-observability split at depth ≥2. The gate model must exceed both the fine-tuned model and the symbolic tracker in accuracy by a margin outside run-to-run variance. It must also produce calibrated belief probabilities, measured by expected calibration error, where the symbolic tracker cannot. On the explicit split we expect, and would report, parity with the symbolic tracker.
Why not just…?
Why not just fine-tune the language model end to end? This is the most serious threat, given the +27/+29 transfer gains. Our prediction is that it fails in two specific places, and both are tested. The first is depth extrapolation: training at depth ≤2 and testing at depths 3–4. A fixed update rule applied along longer chains should extrapolate, whereas a model that learned surface patterns has no reason to. The second is implicit observability, because the fine-tuning data states witnessing explicitly. If the fine-tuned model matches us on both, the gate adds nothing beyond interpretability, and we would say so.
Why not just parse the story and run a symbolic tracker? On clean stories this likely does work, and we expect it to tie. The proposal's claim stands or falls on the thresholding ablation under graded ambiguity. The Bayes argument predicts the soft gate's advantage grows with cue ambiguity and with depth, because rounding errors multiply along the chain. If a well-tuned threshold matches the soft gate across ambiguity levels, the contribution reduces to the benchmark variant plus a negative result, and we will report it that way.
Why not just prompt the model to reason about each character's perspective? Prompting leaves the model to resolve observation and update jointly in free text, which is where ExploreToM locates the failure: state tracking breaks and belief tracking breaks with it. EnactToM makes the same point from the action side. Gemini-Pro answered 63.3% of literal belief probes correctly but succeeded on only 6.7% of functional tasks, and all seven frontier models scored 0.0% Pass^3 on the Hard split (EnactToM). The prompted baseline above measures this directly under our conditions.
Why not a graph attention network over an entity graph? Attention over agents and objects would learn some mixing, but with no constraint tying the weight to observation. The graph literature suggests unconstrained aggregation defaults to smoothing toward neighbors, which here means toward reality. The "text-only gate" and "no second-order terms" ablations bracket how much structure is needed.
Proposed Experiments
The first experiment, on ExploreToM explicit and implicit splits, reports accuracy and expected calibration error by nesting depth and ambiguity level. It also includes zero-shot transfer to ToMi and Hi-ToM, compared against the reproduced fine-tuned model. The decisive check is the interaction plot: the soft-minus-hard gap should be near zero on explicit stories and rise monotonically with ambiguity and depth. If the gap is flat, the core claim is wrong.
The second experiment tests depth composition. We compare the observed error at depths 3–4 against the error predicted by composing per-gate errors. This reveals whether the learned correlation term is needed, and whether nested recursion degrades faster than the product predicts. It is the belief-tracking counterpart of the depth limits studied in AIR and Optimality of Message-Passing Architectures.
The third experiment is adversarial cues. We insert misleading witness cues, for example a character described near a room who cannot actually see into it. This measures whether the gate overtrusts cues, the failure mode flagged by the clean-features assumption in node feature kernels.
The fourth experiment is an embodied stretch on EnactToM's 300 tasks. It replaces the language model's messaging decisions with discrepancy-ranked messages from the gate while keeping the same planner. It reports Pass^3 on the Standard and Hard splits against the 0.0% Hard-split reference, along with the rate of the two documented failure modes. A clear reduction in "withholding critical information" with unchanged navigation success would be direct evidence that a gated belief model narrows the saying–doing gap. No change would locate the bottleneck elsewhere.
References
- Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework
- Model Degradation Hinders Deep Graph Neural Networks
- Optimality of Message-Passing Architectures for Sparse Graphs
- A Logical Analysis of Hanabi
- Reasoning about only Knowing with Many Agents
- Explore Theory of Mind: program-guided adversarial data generation for theory of mind reasoning
- EnactToM: An Evolving Benchmark for Functional Theory of Mind in Embodied Agents
- Node Similarity Preserving Graph Convolutional Networks
- Node Feature Kernels Increase Graph Convolutional Network Robustness
- The Devil is in the Conflict: Disentangled Information Graph Neural Networks for Fraud Detection