Abstract
We show that explicit pragmatic inference aids in correctly generating and following natural language instructions for complex, sequential tasks. Our pragmatics-enabled models reason about why speakers produce certain instructions, and about how listeners will react upon hearing them. Like previous pragmatic models, we use learned base listener and speaker models to build a pragmatic speaker that uses the base listener to simulate the interpretation of candidate descriptions, and a pragmatic listener that reasons counterfactually about alternative descriptions. We extend these models to tasks with sequential structure. Evaluation of language generation and interpretation shows that pragmatic inference improves state-of-the-art listener models (at correctly interpreting human instructions) and speaker models (at producing instructions correctly interpreted by humans) in diverse settings.
Introduction
Most standard NLP models for instruction following (listening) or generation (speaking) are "literal." A literal listener maps text to actions based on statistical patterns, and a literal speaker produces the most likely description for a path. However, human communication is inherently pragmatic. We don't just say what is true; we say what is useful and unambiguous.
For example, if there are two similar paths, a human speaker will mention a landmark to distinguish them. Conversely, a human listener uses counterfactual reasoning: "If the speaker wanted me to go to the very end of the hall, they would have said 'go to the end'; since they didn't, they probably mean this closer turn." The challenge addressed here is how to implement this "Theory of Mind" style reasoning in deep learning models for complex, sequential tasks where the space of possible actions and sentences is massive.
Methods: The Rational Speech Acts Framework
The authors build on the Rational Speech Acts (RSA) framework. They start with "base" models ($L_0$ for the listener and $S_0$ for the speaker), which are standard attentional LSTM sequence-to-sequence models. They then layer pragmatic reasoning on top:
- The Rational Speaker ($S_1$): To describe a path, it generates several candidate instructions using its base model. It then "simulates" a listener ($L_0$) to see which candidate is most likely to lead that listener to the correct destination. It chooses the instruction that is most informative and least ambiguous.
- The Rational Listener ($L_1$): When it receives an instruction, it considers several possible action sequences (paths). It then "simulates" a speaker ($S_0$) to see which path would most likely have prompted the speaker to use those specific words. This allows the listener to resolve ambiguities by thinking about what the speaker didn't say.
Evaluation and Results
The models were evaluated on four distinct domains: SAIL (navigation), and three SCONE domains (Alchemy, Scene, and Tangrams). These domains are particularly challenging because they require resolving context-dependent language like "the other one" or "undo the last step."
The pragmatic models outperformed literal base models and previous state-of-the-art systems across the board:
- Instruction Following: On the SAIL dataset, the rational listener improved accuracy by up to 10.9% in multi-sentence settings.
- Instruction Generation: When human workers followed the AI's instructions, success rates jumped by 12% to 46% compared to the base models.
- Surpassing Humans: In the Tangrams domain, the pragmatic speaker actually produced instructions that were easier for humans to follow (88% success) than the original human-generated instructions in the dataset (66% success).
Discussion and Insights
- The "Aha!" of Counterfactuals: The rational listener's ability to stop early (as seen in Figure 1b) is a direct result of reasoning that "if the speaker wanted me to go further, they would have mentioned the sofa."
- BLEU vs. Utility: A major insight is that pragmatic speakers often produce instructions that have lower BLEU scores (similarity to human-written text) but are much easier for humans to actually follow. This suggests that "human-like" text isn't always the most functional.
- Limitations: The pragmatic inference procedure is computationally expensive, requiring the generation and scoring of many candidates. Furthermore, the pragmatic model is limited by the quality of candidates provided by the base model.
Summary updated