That’s Optional: A Contemporary Exploration of “that” Omission in English Subordinate Clauses

Ella RabinovichACL 2024

TL;DR

This paper explores why English speakers sometimes omit the word "that" when introducing subordinate clauses, a phenomenon known as "that"-omission. Using a large dataset of online English text and advanced language models, the authors demonstrate that speakers optimize for uniform information density, making sentences easier to process by avoiding unexpected information spikes.

What Problem Does This Paper Address?

In English, subordinate clauses can often be introduced by "that" (e.g., "I think that it will rain") or have "that" omitted (e.g., "I think it will rain"). While this omission is common, the linguistic factors influencing it are complex. Previous research has suggested that speakers aim for a relatively uniform "information density" throughout their utterances, meaning they try to avoid sudden increases or decreases in how much information each part of a sentence conveys. This paper investigates whether this Uniform Information Density (UID) hypothesis can computationally explain when speakers choose to omit "that" in subordinate clauses, extending prior work with more modern tools and a larger dataset.

Methods

The authors leverage contemporary Large Language Models (LLMs), specifically the OPT-125m model, to estimate two key information-theoretic measures:

  • SC onset surprisal: This measures how predictable the first word of a subordinate clause is, given the preceding main clause. Higher surprisal means the word is less predictable and thus carries more information. The paper refines the calculation of surprisal to account for the presence or absence of "that" itself.
  • SC onset entropy: This measures the overall uncertainty or variability in predicting the first word of a subordinate clause, considering all possible continuations.

These LLM-derived measures are then used as predictors in a logistic regression model to determine their influence on "that" omission. Other predictors, such as main clause length, subordinate clause length, and main verb frequency, are also included to provide a comprehensive analysis.

Datasets

The primary dataset for this study was collected from Reddit discussions. This platform was chosen because its communication style often blends informal speech with written text, making it a suitable proxy for spontaneous language. The authors extracted approximately 487,614 sentences, categorizing them into those with explicit "that" conjunctions, implicit "that" conjunctions (where it was omitted), and other uses of "that." A subset of 500 sentences was manually evaluated to ensure the accuracy of the extraction process.

Key Insights and Ideas

  • UID as a Driving Force: The study strongly supports the UID hypothesis, showing that speakers' choices about "that" omission are significantly influenced by the need to maintain uniform information density.
  • Surprisal and Entropy Matter: Both SC onset surprisal and SC onset entropy, as computed by LLMs, are powerful predictors of "that" omission. Higher surprisal and entropy in the subordinate clause onset are associated with a greater likelihood of omitting "that."
  • Complementary Predictors: While related, SC onset surprisal and entropy were found to be largely uncorrelated in this dataset, suggesting they capture distinct but complementary aspects of information processing that influence syntactic choices.
  • Beyond Simple Frequency: While main verb frequency is a factor, the study highlights that information-theoretic measures like surprisal offer additional predictive power, suggesting that the predictability of the upcoming information is more crucial than just how often a verb appears.
  • LLMs as Powerful Tools: The use of LLMs for estimating surprisal and entropy demonstrates their utility in computational linguistics for analyzing nuanced linguistic phenomena.

Key Results

The logistic regression model achieved significant classification accuracy:

  • 0.63 accuracy when considering all main verb lemmas.
  • 0.88 accuracy when focusing specifically on sentences with the main verb "think."

This indicates that the proposed predictors, particularly SC onset surprisal and entropy, are highly effective in distinguishing between "that"-omitted and "that"-explicit sentences. The study also found that longer main and subordinate clauses increase the likelihood of "that" omission, a finding consistent with prior research.

Limitations and Future Work

The authors acknowledge several limitations:

  • Restrictive Sentence Setup: The study primarily focused on sentences where the SCONJ "that" immediately follows the main verb. Cases where other words intervene between the main verb and the subordinate clause (e.g., adverbs) were not deeply investigated.
  • Relative Clauses Excluded: Sentences where "that" functions as a relative conjunction (e.g., "the book that I read") were excluded due to their lower frequency and different grammatical function.

Future work could extend the analysis to these more complex sentence structures and explore the interaction of UID with other linguistic theories and factors influencing complementizer omission. The authors also suggest further investigation into the relationship between surprisal and frequency.

Summary updated

BibTeX
@inproceedings{rabinovich2024thats,
  author = {Ella Rabinovich},
  title = {{That’s Optional: A Contemporary Exploration of “that” Omission in English Subordinate Clauses}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure