TL;DR
This paper introduces a "Model Synthesis Architecture" (MSA) that models human open-world cognition by dynamically generating small, tailored probabilistic models on-demand. The MSA, combining language models for relevance retrieval and probabilistic programs for reasoning, better captures human judgments than pure language models, suggesting a path towards more robust and coherent AI reasoning.
What Problem Does This Paper Address?
Humans excel at reasoning in novel, "open-world" situations, drawing on vast background knowledge to make coherent inferences and predictions. Traditional AI models, including specialized probabilistic models, are often limited to the scope of their training data or pre-defined domains. Large Language Models (LLMs) show promise in accessing broad knowledge but struggle with the coherent, structured reasoning that humans exhibit. This paper tackles the challenge of how humans achieve this flexible yet coherent reasoning, hypothesizing that we construct small, ad-hoc mental models tailored to the specific situation at hand. The goal is to build AI systems that can similarly synthesize these bespoke models to reason effectively in novel and complex environments.
Methods
The core of the proposed solution is the Model Synthesis Architecture (MSA). It operates in a staged process:
- Parsing: Natural language inputs (background, observations, questions) are parsed into candidate probabilistic program expressions using a language model.
- Informal Background Knowledge Retrieval: The language model retrieves and synthesizes relevant background knowledge, expanding on the initial input.
- Conceptual Dependency Graph Generation: A graph representing the relationships between variables is generated.
- Probabilistic Model Synthesis: A formal symbolic probabilistic model (a probabilistic program) is synthesized, integrating the retrieved knowledge and structured dependencies. This is the "ad-hoc mental model."
- Probabilistic Inference: Bayesian inference is performed within the synthesized model to answer the queries.
The MSA leverages a large language model (Llama-3.1-70B) for natural language understanding, generation, and guiding the synthesis process, and a probabilistic programming language (WebPPL) for representing and reasoning with the synthesized models.
Datasets
The authors introduce a new benchmark called the "Model Olympics" domain. This domain consists of natural language vignettes about sporting events designed to test open-ended reasoning. It includes:
- Exp. 1: Detailed backgrounds specifying causal variables, their forms, and dependencies.
- Exp. 2: Underspecified backgrounds that name variables but require participants/models to retrieve functional forms and dependencies from background knowledge.
- Exp. 3: Underspecified backgrounds augmented with participant-generated "commentary" introducing novel variables and dependencies, pushing towards truly open-world reasoning.
The vignettes span three sports: tug-of-war, canoe racing, and biathlon, each with distinct causal structures.
Key Insights and Ideas
- On-Demand Model Synthesis: The core idea is that human cognition doesn't rely on a single, monolithic world model but dynamically constructs small, situation-specific models. This "on-demand" synthesis allows for flexibility and efficiency.
- Bridging Neural and Symbolic: The MSA effectively bridges the gap between the broad knowledge retrieval capabilities of neural language models and the structured, coherent reasoning of symbolic probabilistic models.
- LLMs as Knowledge Retrievers, PPLs as Reasoners: The MSA positions LLMs as powerful tools for retrieving and structuring relevant information, while probabilistic programs provide the formal framework for robust inference.
- Human Reasoning is Probabilistic and Structured: The results suggest that human judgments, even in novel situations, are well-aligned with probabilistic inference over structured, albeit dynamically generated, mental models.
- LLMs Struggle with Coherent Integration: While LLMs can access knowledge, they often lack the inherent coherence constraints of probabilistic models, leading to less robust reasoning, especially in open-world scenarios.
Key Results
-
The MSA significantly outperforms pure LLM baselines (direct and chain-of-thought) in capturing human judgments across all three experiments, particularly in the more open-ended Exp. 3.
-
Correlational analyses (R²) show that the MSA's predictions are generally well-aligned with human judgments, often falling within the range of human-human agreement.
- Distributional analyses (Wasserstein Distances) further confirm that the MSA captures not just the average human judgment but also the distribution of predictions, mirroring human uncertainty.
- In Exp. 3, where novel variables were introduced, the MSA showed a marked advantage over LLM baselines, demonstrating its ability to generalize and adapt to unseen situations.
- Qualitative analysis of synthesized models shows that the MSA can retrieve and formalize relevant background knowledge and causal dependencies from natural language inputs.
Limitations and Future Work
- Model Diversity: The authors acknowledge that their MSA implementation sometimes generates models with less diversity than human judgments, potentially leading to lower variance in predictions. Future work aims to increase model diversity.
- Prompt Sensitivity: The synthesis process can be sensitive to prompt design. Future work will explore more generalizable prompting strategies or fine-tuning models to reduce reliance on specific examples.
- Handling Novel Primitives: The MSA sometimes struggles to invent entirely new probabilistic primitives when needed, relying on existing ones.
- Fine-grained Distributional Features: Capturing the "peakier" opinions seen in human judgments, as opposed to smoother distributions, remains an area for future research.
- Broader Domains and Synthesis Strategies: The authors suggest exploring other modeling languages, multi-agent reasoning, and more sophisticated model synthesis strategies (e.g., learning from feedback, considering multiple models). They also propose evaluating MSAs based on ground truth accuracy and their ability to capture human cognitive dynamics.
Summary updated