Uniform information density explains subject doubling in French

Yiming Liang, Pascal Amsili, Heather Burnett, Vera DembergSPIRE 2024 · OpenAlex

Abstract

In this paper, we investigate whether subject doubling in French is affected by the Uniform Information Density (UID) principle, which states that speakers prefer language encoding that minimizes fluctuations in information density. We show that, other factors being controlled, speakers are more likely to double the NP subject when it has a high surprisal, thus providing further empirical evidence to the UID principle which predicts a surprisal-redundancy trade-off as a property of natural languages. We argue for the importance of employing GPT-2 to investigate complex linguistic phenomena such as subject doubling, as it enables the estimation of subject surprisal by considering a rather large conversational context, a task made possible by powerful language models that incorporate linguistic knowledge through pre-training on extensive datasets.

Introduction

In spoken French, speakers frequently employ "subject doubling" (SD), a construction where a nominal subject is immediately followed by a redundant clitic pronoun (e.g., Marie elle veut une pomme instead of Marie veut une pomme). While traditionally analyzed through the lens of social variation or syntax, this study explores SD as a cognitive strategy driven by the Uniform Information Density (UID) hypothesis. UID suggests that human communication is optimized when information is distributed evenly across an utterance, avoiding processing bottlenecks caused by sudden "spikes" in information.

Under this framework, subject doubling acts as a smoothing mechanism. By adding a redundant pronoun, a speaker can reduce the local information density following a highly unpredictable or "surprising" subject, thereby facilitating easier processing for the listener.

Methods

To test the relationship between predictability and doubling, the researchers quantified the "surprisal" of noun phrase (NP) subjects within real-world dialogue. Surprisal is defined as the negative log probability of a word given its preceding context: $-\log P(\text{word} | \text{context})$.

The study utilized GPTfr-124M, a French variant of GPT-2, to estimate these probabilities. Unlike traditional n-gram models, GPT-2 can process large conversational windows, allowing it to approximate the "world knowledge" and discourse context available to human speakers. To ensure the model captured the nuances of informal speech, it was fine-tuned on the Corpus de Français Parlé Parisien des années 2000 (CFPP2000), a 750,000-token dataset of spoken Parisian French.

Example of surprisal calculation for a noun phrase subject given a conversational context.

The researchers then applied a mixed-effects logistic regression to the Multicultural Parisian French (MPF) corpus (approx. 790,000 words). This model tested whether surprisal predicted the presence of doubling while controlling for confounding factors such as verb frequency, subject length, and sentence polarity.

Key Results

The statistical analysis revealed a robust correlation between information density and syntactic choice. Higher surprisal of the NP subject significantly increased the likelihood of subject doubling ($p < 0.001$).

As shown in the data, shifting from the minimum observed surprisal to the maximum increased the average probability of doubling from 0.71 to 0.87. The final predictive model, incorporating both surprisal and control factors, achieved an accuracy of 86.68% in identifying whether a speaker would utilize doubling in a given context.

Proportion of subject doubling as a function of NP subject surprisal.

These findings suggest that the cognitive pressure to maintain uniform information density is a robust driver of speech planning. The effect remained significant even when the model was provided with varying amounts of preceding context (from 1 to 11 turns), indicating that the UID principle operates consistently across different levels of discourse depth.

Discussion and Conclusion

This study provides strong empirical support for the UID principle within the context of French morphosyntax. It demonstrates that redundancy in natural language is not merely "noise" or a quirk of informal register, but a functional tool for information smoothing.

Furthermore, the research highlights the utility of Large Language Models (LLMs) as proxies for human cognitive processing in psycholinguistics. By leveraging the pre-trained linguistic knowledge of models like GPT-2, researchers can investigate complex phenomena that were previously difficult to quantify. Future work may explore the intersection of mathematical surprisal and discourse-level "topicalization," as well as the application of this methodology to other languages featuring redundant syntactic markers.

Summary updated

Abstract

In this paper we investigate whether subject doubling in French is affected by the Uniform Information Density (UID) principle, which states that speakers prefer language encoding that minimizes fluctuations in information density. We show that, other factors being controlled, speakers are more likely to double the NP subject when it has a high surprisal, thus providing further empirical evidence to the UID principle which predicts a surprisal-redundancy trade-off as a property of natural languages. We argue for the importance of employing GPT-2 to investigate complex linguistic phenomena such as subject doubling, as it enables the estimation of subject surprisal by considering a rather large conversational context, a task made possible by powerful language models that incorporate linguistic knowledge through pre-training on extensive datasets.
BibTeX
@inproceedings{liang2024uniform,
  author = {Yiming Liang and Pascal Amsili and Heather Burnett and Vera Demberg},
  title = {{Uniform information density explains subject doubling in French}},
  booktitle = {SPIRE},
  year = {2024},
}
Figure