The Effect of Word Predictability on Spoken Cross-Language Intelligibility

Wei Xue, Iuliia Zaitova, Bernd Möbius2025 · DOI

TL;DR

This paper investigates how the predictability of a word within a sentence affects how well speakers of one language can understand spoken words in another language (cross-language intelligibility). The study found that word predictability significantly impacts this understanding, and this effect appears to be language-specific, with English speakers benefiting more from written context than German speakers.

What Problem Does This Paper Address?

Receptive multilingualism (RM) is the ability to understand a language without prior learning. While factors like linguistic distance and speaker attitude are known to influence RM, the role of word predictability—how likely a word is to appear given its context—has been largely unexplored. This is surprising because word predictability is known to aid comprehension and speech perception in native language processing. This paper aims to fill this gap by experimentally examining how word predictability affects spoken cross-language intelligibility, specifically for Dutch words understood by German and English native speakers. It also explores whether providing written context alongside audio can improve this intelligibility.

Methods

The researchers designed a free translation experiment using Dutch spoken sentences. They created stimuli where target Dutch words (cognates across Dutch, German, and English) were embedded in sentences with either high or low word predictability. This predictability was quantified using trigram surprisal values derived from monolingual language models.

Two experimental conditions were used:

  • AudioOnly: Participants only heard the spoken Dutch sentences.
  • AudioText: Participants heard the sentences and also saw the written text.

Participants were asked to translate the target cognate word at the end of each sentence. Statistical analysis was performed using Generalized Linear Mixed-Effect Models (GLMMs) to assess the impact of word predictability, trigram surprisal, and experimental condition on translation accuracy.

Datasets

The study utilized existing monolingual trigram language models trained on:

  • CGN [14] for Dutch
  • ukWaC for English
  • deWaC for German

These models were used to extract trigram surprisal values. Word lemma frequencies were sourced from CELEX [10] and SUBTLEX [11, 12, 13]. The stimuli were validated by Dutch native speakers.

Key Insights and Ideas

  • Predictability Matters, Even Across Languages: Contrary to initial expectations based on stimulus validation, word predictability significantly influences cross-language intelligibility. This suggests that the cognitive mechanisms that leverage context for native language comprehension may extend, to some degree, to cross-language understanding.
  • Language-Specific Effects: The impact of word predictability and the benefit of written context differ between German and English speakers. This highlights that cross-language intelligibility is not a universal phenomenon but is shaped by the linguistic background of the listener.
  • The Double-Edged Sword of Predictability: While high trigram surprisal and low word predictability separately seemed to increase accuracy in some analyses, their interaction strongly decreased accuracy. This suggests that when both factors are extreme (high surprisal and low predictability), they can interfere with comprehension, rather than aid it.
  • AudioText Advantage for English Speakers: English speakers showed a significant benefit from the AudioText condition, indicating that written context is crucial for their cross-language comprehension, likely due to greater linguistic distance from Dutch. German speakers, whose languages are more closely related to Dutch, did not show a significant benefit from the written text.

Key Results

  • Baseline Accuracy: Baseline accuracy was low for both groups, suggesting that cross-language intelligibility is challenging without additional cues. German speakers generally performed better than English speakers.
  • Predictability Effects: For German speakers, high trigram surprisal and low word predictability led to significant positive estimates for correct translations. However, the interaction of high surprisal and low predictability resulted in a significant decrease in accuracy.
  • English Speakers and AudioText: For English speakers, high trigram surprisal and low word predictability also led to significantly higher accuracy, but with weaker effects than in German. Crucially, the AudioText condition resulted in significantly higher accuracy compared to AudioOnly.
  • Interaction Effect: A significant interaction between trigram surprisal and word predictability was observed in both groups, but with a less severe effect in English speakers. This interaction generally led to lower accuracy when both high surprisal and low predictability were present.

Limitations and Future Work

The authors acknowledge that their GLMM findings did not perfectly align with initial stimulus validation predictions. They also note that the manipulation of trigram surprisal was intertwined with syntactic structure (prepositional phrases vs. noun phrases), which could have influenced the results.

Future research directions suggested include:

  • Disentangling the effects of trigram surprisal from syntactic structures.
  • Investigating the influence of additional factors like L2 proficiency, phonetic distances, and cultural context.
  • Using stimuli validated across all three languages.
  • Exploring the role of participant age, hearing ability, and cognitive speed, as younger participants might be more adaptable and older participants might rely more on native language cues.

Summary updated

BibTeX
@misc{xue2025effect,
  author = {Wei Xue and Iuliia Zaitova and Bernd Möbius},
  title = {{The Effect of Word Predictability on Spoken Cross-Language Intelligibility}},
  year = {2025},
  doi = {10.21437/interspeech.2025-1676},
}
Figure