Abstract
Knowledge organizes our understanding of the world, determining what we expect given what we have already seen. Our predictive representations have two key properties: they are productive, and they are graded. Productive generalization is possible because our knowledge decomposes into concepts—elements of knowledge that are combined and recombined to describe particular situations. Gradedness is the observable effect of accounting for uncertainty—our knowledge encodes degrees of belief that lead to graded probabilistic predictions. To put this a different way, concepts form a combinatorial system that enables description of many different situations; each such situation specifies a distribution over what we expect to see in the world, given what we have seen. We may think of this system as a probabilistic language of thought (PLoT), in which representations are built from language-like composition of concepts, and the content of those representations is a probability distribution on world states. The purpose of this paper is to formalize these ideas in computational terms, to illustrate key properties of the PLoT approach with a concrete example, and to draw connections with other views of conceptual structure.
Introduction
Human cognition possesses two seemingly contradictory traits: productivity and gradedness. Productivity allows us to combine simple concepts (like "strength" and "team") to understand an infinite variety of new situations, such as a 100-person tug-of-war. Gradedness allows us to reason under uncertainty, maintaining degrees of belief rather than binary "true/false" certainties.
Traditional AI approaches usually excel at one but fail at the other. Symbolic models, such as classical logic, are highly productive but brittle when faced with noise. Conversely, standard probabilistic models, like Bayesian Networks, handle uncertainty well but are often restricted to a fixed, finite set of variables, making them difficult to scale to the open-ended complexity of human thought. This paper proposes the Probabilistic Language of Thought (PLoT) hypothesis, which defines concepts as stochastic functions within a probabilistic programming language, seeking a "marriage" of these two paradigms.
The PLoT Framework
The authors formalize the PLoT using Church, a probabilistic programming language based on the $\lambda$-calculus. In this framework, concepts are treated as reusable pieces of code that return values drawn from a distribution. For example, a "strength" concept is a function that draws from a Gaussian distribution.
Complex ideas are built through compositionality, nesting these functions. A "team strength" concept is simply a function that sums the outputs of individual "player strength" functions. To ensure consistency—for instance, ensuring a player's inherent strength doesn't change mid-match—the framework utilizes memoization (mem), which stores the result of a function call for a specific input.
Reasoning is treated as Bayesian inference. The model uses a query function to work backward from observed outcomes (e.g., who won a game) to hidden variables (e.g., how strong is the player?). Furthermore, for physical reasoning, the authors employ a "condition-intervene-predict" cycle to handle counterfactuals: conditioning on what happened, intervening by changing a variable, and re-running the simulation to observe the outcome.
Experimental Validation
The PLoT hypothesis was validated by comparing model predictions against human behavioral data from two experiments involving a ping pong tournament scenario.
In Experiment 1, 30 participants judged the strength of players in various tournament formats (single and two-player matches) with different patterns of evidence, including confounded, indirect, and diverse evidence. Experiment 2 involved 20 participants performing the same task but with additional "omniscient" information from a commentator (e.g., "Player B was lazy in game 2").
The interface shown above demonstrates how participants were presented with evidence (match winners) and asked to provide a graded judgment of a specific player's strength. This setup directly tests the model's ability to perform inference over compositional structures.
Key Results and Insights
The PLoT model showed a remarkable fit with human intuition. In Experiment 1, the model's strength predictions correlated with human judgments at $r = 0.98$. In Experiment 2, the correlation remained high at $r = 0.97$.
The model successfully captured "indirect evidence." For example, if player A beats B and B beats C, the model (and humans) correctly infers that A is likely very strong, even if A never played C. This demonstrates the power of viewing concepts as programs; the model doesn't need a new structure for every possible match-up but simply calls the same functions with different arguments.
In the physics domain, the model correctly predicted that human causal judgments are based on comparing actual events to "noisy" counterfactual simulations. This suggests that human reasoning is akin to running mental simulations of possible worlds consistent with observations.
Conclusion and Future Work
While the PLoT framework provides a powerful computational level of analysis, it faces challenges regarding computational cost. Performing inference over complex programs is expensive, suggesting that the brain likely employs shortcuts like caching or optimized Monte Carlo simulations.
Future work remains in the area of concept acquisition or "program induction"—learning a new concept by writing a new function. Additionally, the neural grounding of these stochastic functions remains an open question. Despite these challenges, the PLoT framework offers a robust path toward understanding the compositional and probabilistic nature of human thought.
Summary updated