Abstract
When people answer questions about a specific situation, cognitive science suggests that they form a mental picture of that situation before answering. While we do not know how language models (LMs) answer such questions, we conjecture that they may answer more accurately if they are also provided with additional details about the question situation, elaborating the "scene." To test this conjecture, we train a new model, DREAM, to answer questions that elaborate the scenes that situated questions are about, and then provide those elaborations as additional context to a question-answering (QA) model. We find that DREAM is able to create better scene elaborations (more accurate, useful, and consistent) than a representative state-of-the-art, zero-shot model (Macaw). We also find that using the scene elaborations as additional context improves the answer accuracy of a downstream QA system, including beyond that obtainable by simply further finetuning the QA system on DREAM's training data. These results suggest that adding focused elaborations about a situation can improve a system's reasoning about it, and may serve as an effective way of injecting new scenario-based knowledge into QA models.
Introduction
Language models (LMs) often excel at multiple-choice benchmarks, yet it remains unclear whether they possess a robust understanding of underlying social dynamics or are merely relying on surface-level pattern matching. For example, judging the morality of an action requires inferring intent and predicting outcomes. While humans do this instinctively, LMs often struggle to articulate these hidden dimensions. This research investigates whether explicitly teaching a model to "elaborate the scene" before answering can bridge this gap in reasoning.
Methods: The DREAM Framework
The authors propose a modular, two-step pipeline designed to mimic human mental modeling:
- Scene Elaboration (SE): Given a situation, the DREAM (Dynamically wRitten ElAborations to iMprove question-answering) model generates a four-dimensional elaboration:
- Motivation (M): What drove the character's actions?
- Emotion (E): How do the characters feel afterward?
- Rule of Thumb (ROT): What is the relevant social norm?
- Consequence (Con): What is likely to happen next?
- Question Answering (QA): This generated text is prepended to the original situation and fed into an off-the-shelf, frozen QA model (such as Macaw or UnifiedQA) to predict the final answer.
DREAM was developed by fine-tuning a T5-11B model using distant supervision from existing commonsense datasets, including Story Commonsense (for M and E), Social Chemistry (for ROT), and Moral Stories (for Con).
Key Results
Superior Scene Elaboration
DREAM significantly outperforms the baseline Macaw model in generating high-quality mental models. Human evaluators rated DREAM's outputs as substantially more accurate, useful, and consistent across all tested datasets, including ETHICS-CS, CODAH, and Social IQA.
Impact on Downstream QA
Adding DREAM's elaborations as context improved zero-shot QA performance across the board. For instance, on Social IQA, accuracy jumped from 64.84% to 69.06%. The approach is model-agnostic and particularly effective for smaller models; Macaw-3B saw more significant absolute gains than larger models, suggesting that explicit scene elaboration can compensate for lower parameter counts.
The Value of Holistic Context
Ablation studies confirmed that providing all four dimensions (Motivation, Emotion, Rule of Thumb, and Consequence) yields better QA performance than providing any single dimension alone. This suggests that a holistic "mental picture" is more effective for reasoning than isolated pieces of situational context.
Discussion and Limitations
Despite its success, DREAM is not without limitations. Approximately 12% of its generated components contain hallucinations, and in some cases, downstream QA models fail to exploit even highly accurate context. Future work will explore dynamic selection of elaboration dimensions and further refining the integration between the elaboration and reasoning components.
Conclusion
DREAM demonstrates that explicitly uncovering the "mental models" behind situational questions can significantly enhance LM reasoning. By modularizing scene elaboration, this approach provides a portable and interpretable method for improving performance on complex social and ethical tasks without requiring retraining of the primary reasoning model.
Summary updated