TL;DR
This paper investigates why people interpret language differently, specifically focusing on "scalar implicatures" (like inferring "not all" from "some"). They propose that this variation stems from a listener's uncertainty about the possible meanings of a word in a given context, rather than just how predictable a stronger meaning is. Their findings suggest that pragmatic inferences are driven by context-dependent uncertainty over alternatives.
What Problem Does This Paper Address?
When someone says "some students passed the exam," we often infer that not all students passed. This is a scalar implicature (SI). While this inference seems straightforward, empirical studies show that people don't always make it, and the rate at which they do varies significantly across different words and contexts. This "scalar diversity" is a puzzle: what factors determine whether a listener will draw such an inference? Previous research has explored factors like the "availability" of the stronger meaning (e.g., how easily we can think of "all" after hearing "some"). However, this paper argues that previous operationalizations of "availability" might be insufficient. They propose a new hypothesis: SI strength depends on a listener's uncertainty about the underlying scale of meanings associated with a word in a specific context.
Methods
The researchers use a fine-tuned T5 language model to estimate human-like predictions about language. They employ two main approaches to quantify the "alternatives" a listener might consider:
- String-based measures: They treat the actual words or phrases that could complete a sentence as alternatives. They use the T5 model to predict the probability of different strings filling a masked position in a "scalar construction" (e.g., "X, but not Y").
- Surprisal: They measure how predictable a strong scalemate is by calculating its surprisal (negative log probability) within the T5 model.
- Scale Uncertainty: They measure uncertainty by calculating the entropy over the distribution of predicted alternative strings.
- Hierarchical (Conceptual) measures: They acknowledge that listeners might reason about concepts rather than just surface strings. They use Sentence-T5 embeddings to represent the meaning of alternatives.
- Class-based Surprisal: They cluster these embeddings into latent classes and measure the surprisal of the class containing the strong scalemate.
- Class-based Scale Uncertainty: They calculate the entropy over the probabilities of these latent classes.
Datasets
The study uses data from Experiment 2 of van Tiel et al. (2016), which collected human judgments on scalar implicature rates for 32 adjectival scales. The T5 model was fine-tuned on a 10-million-token subset of the 2007 portion of the New York Times Corpus.
Key Insights and Ideas
- Uncertainty is Key: The most significant finding is that a listener's uncertainty about the possible meanings (alternatives) of a word in a given context is a strong predictor of scalar implicature strength. Lower uncertainty leads to stronger inferences.
- Predictability of Strong Scalemate Isn't Enough: Contrary to some prior assumptions, the simple predictability of the stronger meaning (e.g., how likely "all" is to follow "some") does not reliably predict SI rates. This suggests that the ease of retrieving a strong meaning might be less important than the listener's confidence in which meaning is intended.
- Conceptual vs. String-based: Both string-based and conceptual (embedding-based) measures of uncertainty show similar predictive power, suggesting that both surface form and underlying meaning play a role in how listeners process scalar inferences.
- Context Matters: The paper emphasizes that the "scale" and its associated alternatives are not fixed but are dynamically constructed based on the context. This context-driven uncertainty is what drives pragmatic reasoning.
Key Results
- String-based Scale Uncertainty: Was a significant predictor of SI rates (p = 0.001). Higher entropy (more uncertainty) in predicted alternative strings correlated with lower SI rates.
- Class-based Scale Uncertainty: Was also a significant predictor of SI rates (p = 0.01). Higher entropy over latent classes correlated with lower SI rates.
- String-based Surprisal: Was not a significant predictor of SI rates (p = 0.928).
- Class-based Surprisal: Was not a significant predictor of SI rates (p = 0.186).
- Scale Boundedness: As found in previous work, scale boundedness remained a highly significant predictor of SI rates.
Limitations and Future Work
- Clustering Algorithm and Representation: The authors acknowledge that their findings might depend on the specific clustering algorithm and the chosen conceptual representation (Sentence-T5 embeddings). They plan to explore these aspects further.
- Generalizability to Non-Entailment Scales: The current work focuses on scales where one meaning entails another (e.g., "intelligent" entails "smart"). They suggest their methods could be extended to scales ordered by "ad-hoc relationships" rather than strict entailment.
- Deriving Behavioral Predictions: The approach offers a promising way to derive quantitative behavioral predictions from non-linguistic representations of alternatives, which could be valuable for understanding pragmatic behaviors more broadly.
Summary updated