TL;DR
This paper investigates why English speakers sometimes omit the word "that" in complement clauses (e.g., "I think they're right" vs. "I think that they're right"). They found that "that" is less likely to be used when the upcoming clause is highly predictable, supporting the Uniform Information Density (UID) hypothesis. Crucially, they show that modern neural language models (like GPT-2) provide a more accurate measure of this predictability than traditional methods, leading to better predictions of "that"-omission.
What Problem Does This Paper Address?
Language production is remarkably flexible; we have multiple ways to say the same thing. A key question in psycholinguistics is why we choose one phrasing over another. The Uniform Information Density (UID) hypothesis proposes that speakers aim for a consistent rate of information flow, avoiding both overly predictable (information-poor) and overly surprising (information-rich) stretches of speech. This paper revisits a classic finding: the optional complementizer "that" in English complement clauses is omitted more often when the clause is predictable. The authors aim to refine our understanding of this phenomenon by using a large, modern dataset and advanced methods to measure predictability, addressing limitations of previous studies that used smaller, older datasets and simpler measures.
Methods
The core of the study involves building statistical models to predict whether "that" is used in a complement clause. They compare two main ways of measuring the predictability of the upcoming clause:
- Verb Subcategorization Probability: This is a traditional method based on how often a specific verb (e.g., "think," "say") is followed by a complement clause in a corpus.
- Contextual Word Embeddings: This is a more modern approach using neural language models (specifically GPT-2). These models capture how a word's meaning and syntactic role change based on its context, providing a richer, more nuanced measure of predictability.
They trained binary classifiers to predict "that"-mentioning, controlling for various linguistic factors like verb frequency, subject type, and syntactic priming (the tendency to repeat structures). They also used mixed-effects models to account for variation across individual speakers and verbs.
Datasets
The study primarily uses the CANDOR corpus, a large, contemporary dataset of naturalistic online conversations. This corpus contains over 50,000 instances of complement clauses, significantly larger and more modern than datasets used in previous related work.
Key Insights and Ideas
- UID Holds Up, But How We Measure It Matters: The core finding that predictable clauses lead to "that"-omission is replicated. However, the way predictability is measured is critical.
- Neural Models Capture Deeper Predictability: Contextual word embeddings from GPT-2 provide a more accurate measure of syntactic predictability than traditional verb subcategorization probabilities. This suggests that neural models capture more abstract, verb-independent cues about upcoming sentence structure.
- Verb-Specific Preferences Mask General Effects: Traditional measures based on verb subcategorization probabilities likely conflated general information density effects with idiosyncratic preferences of individual verbs. When these verb-specific preferences were accounted for (using random intercepts), the effect of verb-based information density disappeared, while the embedding-based measure remained significant.
- Multiple Factors Shape "That"-Use: While information density is a key driver, the study confirms that other factors also play a role, including:
- Availability: Lower frequency verbs and clause onsets are associated with more "that"-use.
- Syntactic Priming: Speakers are more likely to use "that" if they or their interlocutor used it in the preceding clause.
- Speaker Commitment: Higher commitment (e.g., with first-person subjects) might lead to less "that"-use.
- Similarity Avoidance: Speakers avoid adjacent "that"s.
- Disfluencies: Filled pauses and repetitions increase "that"-use.
Key Results
- The study successfully replicated the finding that higher information density (as measured by contextual word embeddings) significantly increases the likelihood of using "that" (β = 0.12; p < 0.001).
- Models using contextual word embeddings for predictability achieved better performance (lower AIC/BIC) than those using only verb subcategorization probabilities, especially when accounting for verb-specific variation.
- When verb-specific preferences were controlled for using random intercepts, the effect of verb-based information density on "that"-mentioning became non-significant, while the embedding-based measure remained significant.
Limitations and Future Work
- Data Quality: The conversational transcripts were automatically generated, and dependency structures were derived using automatic parsers, meaning potential transcription and parsing errors exist.
- GPT-2 as a Proxy: GPT-2 was used to estimate online spoken language predictions, but it's primarily trained on written text. Future work could explore models trained on conversational data or specifically for speech.
- Single Language Model: The analysis relied on a single language model architecture. Exploring other models could assess the generalizability of the findings.
- Exhaustive Predictors: While many control variables were included, it's possible that other linguistic predictors could influence "that"-reduction. Future research could investigate additional features. The authors suggest exploring alternative models and speech-specific language models for future work.
Summary updated