Introduction
When a human thinks of an egg, they possess a structured mental image: the yolk is inside the white, and both are encased in a shell. This internal representation allows us to immediately reject a statement like "the yolk surrounds the shell." While Large Language Models (LLMs) are trained on vast amounts of text and can generate fluent descriptions of objects, it remains unclear if they develop similar coherent "mental models" or if they merely possess "fragments of knowledge."
Previous research has shown that LMs struggle with physical commonsense. This paper specifically investigates coherence: if an LM believes "A is above B," does it also consistently believe "B is below A"? The authors argue that without such coherence, LMs cannot truly reason about or interact with the physical world effectively.
Methodology: The ParRoT-Con Framework
To address this incoherence, the researchers developed ParRoT-Con (Parts and Relations of Things with Constraints), a two-step neuro-symbolic approach:
- Probing: The system queries a pre-trained LM (such as GPT-3 or Macaw) with an exhaustive list of True/False questions regarding the relationships between parts of an object (e.g., "In an egg, does the yolk surround the shell?"). This generates a set of "raw" beliefs with associated confidence scores.
- Constraint Reasoning: A neuro-symbolic layer is added on top of the LM's raw predictions. This layer uses a MaxSAT (Maximum Satisfiability) solver to enforce four types of commonsense constraints: symmetry (next-to), asymmetry (part-of), inversion (above/below), and transitivity (inside/inside).
The solver finds the most probable set of beliefs that satisfies all hard constraints, effectively "self-correcting" the LM's logical inconsistencies.
The ParRoT Benchmark
The authors introduced a new benchmark dataset to quantify these mental models:
- ParRoT: Consists of 100 everyday things, ranging from natural objects like "trees" to man-made devices like "flashlights." It includes 2,191 parts and 11,720 relationship tuples.
- Gold Standard: To create the ground truth, human annotators drew "parts graphs" for each object, which were then converted into relationship tuples for model evaluation.
Key Insights and Results
The study revealed several critical findings regarding the current state of LM world-knowledge:
- The "Stochastic Parrot" Reality: LMs are surprisingly poor at spatial reasoning. Base models achieved only 54-59% accuracy on ParRoT—barely outperforming a majority-class baseline.
- Prevalent Incoherence: Models frequently contradict themselves. For example, GPT-3 might claim that a yolk surrounds a shell and vice versa. Conditional constraint violations ranged from 19% to 43%.
- Accuracy Boost via Reasoning: Applying the constraint satisfaction layer (ParRoT-Con) improved the accuracy of mental models by 16-20% and reduced the conditional constraint violation rate to 0%.
- Natural vs. Man-made: LMs generally have better mental models of natural living things (animals, plants) than complex man-made devices (typewriters, computers), likely due to more consistent descriptions in training data.
Conclusion
The results suggest that simply making models larger may not solve the coherence problem. Instead, a hybrid architecture that combines the broad knowledge of an LM with the rigorous logic of a symbolic reasoner is a more promising path toward developing robust "world models." While ParRoT-Con significantly closes the gap, a substantial performance difference remains between LMs and human-level coherence, highlighting the ongoing challenge of teaching machines to understand the physical world.
Summary updated