Empirical evidence of Large Language Model's influence on human spoken communication

Hiromu Yakura, Ezequiel Lopez-Lopez, Levin Brinkmann, Ignacio Serna, Prateek Gupta, Iyad RahwanarXiv 2024 · DOI · arXiv

Abstract

Recent research provides the first large-scale empirical evidence that Large Language Models (LLMs) are fundamentally altering human spoken language. By analyzing over 7 billion words from YouTube and podcasts, researchers identified an abrupt surge in "AI-preferred" vocabulary—such as delve, meticulous, and swift—following the release of ChatGPT. This suggests the emergence of a closed cultural feedback loop where machine-generated linguistic patterns are internalized and broadcast back by humans, potentially leading to linguistic homogenization.

Introduction: The "Delve" Effect

While "lexicographic biases" in AI—the tendency to over-rely on professional or polite terms—have been documented in written academic and student work, their impact on spontaneous spoken communication remained unexplored. Spoken language is typically less filtered than writing; its alteration suggests a deeper internalization of AI-driven linguistic patterns. This study quantifies the "cultural feedback loop," where machines trained on human data broadcast cultural traits back to humans, who then adopt them.

Methodology and Data Scale

The researchers analyzed a massive corpus of 7.35 billion words across two primary datasets: 360,445 YouTube academic talks and 771,591 podcast episodes spanning Business, Education, Religion, Science & Tech, and Sports.

Figure 1: Overview of the study's scope, including the transcription pipeline and the diverse pathways through which AI-driven patterns spread.

To identify AI influence, the team developed a "GPT Score" based on the log-odds ratio of words injected by models like GPT-4 when editing human abstracts. To establish causality, they employed the Synthetic Control Method, comparing the frequency of "treated" words (like delve) against a weighted combination of "donor" words that shared similar pre-ChatGPT trends.

Key Findings: The Rise of AI-Speak

The study found a statistically significant (p < 0.01) and abrupt increase in the usage of top GPT-preferred words immediately following the release of ChatGPT in November 2022.

Figure 5: Analysis of word frequency shifts and ChatGPT preferences. Panel A shows the 'delve' spike relative to a synthetic control; Panel C lists top words preferred by ChatGPT.

Abrupt Linguistic Shifts

The most prominent example is the word delve. In academic YouTube talks, its frequency surged dramatically compared to its synthetic control. Other words like comprehend, boast, swift, and meticulous saw annual growth rates of 25% to 50%.

Domain-Specific Diffusion

The adoption of these patterns is not uniform. STEM-adjacent fields (Science, Tech, Business) showed the most significant increases, suggesting a "two-phase diffusion": AI-isms first gain traction in professional environments with high AI usage before filtering into informal discourse. Conversely, domains like Sports and Religion appear currently insulated from this influence.

Implications and Risks

The authors warn of a "cultural singularity" and linguistic homogenization. If AI favors a narrow set of "socially desirable" words and humans adopt them, linguistic diversity may erode. Furthermore, this creates a risk of "model collapse," where future AI models trained on "contaminated" human data lose the ability to generate diverse or novel content.

Conclusion

This research confirms that we are no longer just users of AI; we are becoming its students. While the study detects the shift, the exact mechanism—whether direct imitation or reliance on AI-generated scripts—remains to be explored. As we change the way we speak, we must consider if we are also changing the way we think.

Summary updated

Abstract

From the printing press to social media, innovations in communication technology have repeatedly reshaped how ideas spread through human culture. Chatbots powered by generative artificial intelligence constitute a new medium, encoding cultural patterns in their neural representations and disseminating them in conversations with hundreds of millions of people. Whether these patterns transmit into human language, and ultimately shape human culture, is a fundamental question. While fully quantifying the causal impact of a chatbot like ChatGPT on human culture is challenging, lexical shifts in human spoken communication may offer an early indicator. Here we show that words preferentially generated by ChatGPT, such as delve, showcase, boast, intricacies and meticulous, increased abruptly in spontaneous human speech. A synthetic-control analysis of 737,083 hours of conversation from 824,634 podcast episodes, screened for unscripted speech, causally links this shift to ChatGPT's release. The measurable influence on spontaneous speech suggests that humans internalize the lexical choices of large language models (LLMs). A preregistered experiment (N = 496) confirms they do, as a brief chatbot interaction led participants to adopt its words as their own, persisting past a distractor task and confirmed in forced lexical choice, indicating entrenchment in the active vocabulary. Together these results show that machines trained on human data now feed their own traits back into human language, integrating LLMs into the ongoing processes of cultural evolution.. This coupling raises concerns about linguistic homogenization and the capacity of a few major AI providers for latent cultural influence at scale.
BibTeX
@misc{yakura2024empirical,
  author = {Hiromu Yakura and Ezequiel Lopez-Lopez and Levin Brinkmann and Ignacio Serna and Prateek Gupta and Iyad Rahwan},
  title = {{Empirical evidence of Large Language Model's influence on human spoken communication}},
  year = {2024},
  eprint = {2409.01754},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.01754},
}
Figure