The Different Roles of Expectations in Phonetic and Lexical Processing

Shiri Lev-Ari, Robin Dodsworth, Jeff Mielke, Sharon PeperkampInterspeech 2019

TL;DR

This paper reveals that our expectations about a speaker's background influence speech processing differently at the phonetic and lexical levels. Crucially, individual differences like working memory and implicit bias play distinct roles in these two processes, challenging the assumption that they are driven by a single mechanism.

What Problem Does This Paper Address?

When we listen to someone speak, we don't just process the raw sounds; we leverage our knowledge about the speaker's background (age, region, ethnicity, etc.) to aid comprehension. This is because these demographic factors often correlate with predictable variations in pronunciation. For instance, a speaker from a particular region might pronounce certain vowels differently than a speaker from another. Listeners intuitively use these "expectations" to make sense of speech.

Previous research has shown that these expectations can influence speech perception in two main ways:

  1. Exaggeration: We might perceive a sound as more extreme than it actually is, aligning it with our stereotype of the speaker's accent. For example, if we expect an Australian accent to have a "higher" vowel, we might perceive it as even higher than it is.
  2. Compensation: We might adjust our perception to "correct" for expected deviations from a standard pronunciation. For example, if we expect a speaker to drop a "t" at the end of a word, hearing "bell" might lead us to interpret it as "belt."

However, most studies have used either phonetic (vowel perception) or lexical (word interpretation) tasks, assuming they tap into the same underlying mechanism. This paper questions this assumption and also explores whether individual differences in cognitive abilities and attitudes affect how susceptible people are to these expectation-driven effects.

Methods

The researchers designed an experiment that combined two types of tasks within a single session to directly compare phonetic and lexical processing:

  • Lexical Interpretation Task: Participants heard sentences containing potentially ambiguous words. They were told the speaker was either from North Carolina (General American accent) or the Southern US. The ambiguity was such that a Southern pronunciation of one word could sound like a different word in the General American accent. Participants' responses indicated whether they interpreted the word according to the General American or Southern pronunciation. This task was designed to elicit compensation.
  • Vowel Matching Task: Participants heard a word with a target vowel and then a continuum of synthesized vowels. They had to indicate which synthesized vowel best matched the vowel they heard. Again, the speaker was presented as either from North Carolina or the Southern US. This task was designed to elicit exaggeration.

To investigate individual differences, participants also completed:

  • Working Memory (WM) Task: A standard operation span task to measure their capacity to hold and process information.
  • Autism-Spectrum Quotient (AQ) Questionnaire: To assess autistic traits, which are associated with detail-focus and reduced social attention.
  • Implicit Bias Task (IAT): A single-category IAT was used to measure implicit attitudes towards Southern US culture.

Datasets

The study did not introduce new datasets. Instead, it used:

  • Synthesized vowel continua: Created to represent the phonetic differences between General American and Southern US accents.
  • Recorded speech stimuli: A speaker with a characteristic Southern accent recorded sentences and words.
  • Standardized questionnaires and tasks: For measuring working memory, autistic traits, and implicit bias.

Key Insights and Ideas

  • Phonetic vs. Lexical Processing are Distinct: The most significant finding is that phonetic and lexical processing are not driven by the same mechanism when it comes to expectations. This challenges a long-standing assumption in the field.
  • Working Memory for Phonetics, Implicit Bias for Lexicon: Individual differences play different roles. Working memory capacity specifically influences how expectations affect phonetic perception (leading to exaggeration), while implicit attitudes towards a speaker's background specifically influence lexical interpretation (leading to compensation).
  • Exaggeration vs. Compensation: The study confirmed that expectations can lead to compensation in lexical tasks (as predicted) but surprisingly found no main effect of speaker identity in the vowel matching task, suggesting that the "exaggeration" effect might be more nuanced or context-dependent than previously thought, though the interaction with WM still points to an expectation effect.
  • Attitudes and Stereotypes: The interaction between implicit attitudes and speaker identity in the lexical task suggests that less positive attitudes towards a group might lead to stronger adherence to stereotypes, influencing how we interpret ambiguous speech.

Key Results

  • Lexical Interpretation: Participants were more likely to interpret ambiguous words in line with a General American pronunciation when the speaker was presented as being from Ohio (GA) compared to South Carolina (Southern). This demonstrates a compensation effect. Furthermore, this compensation effect was modulated by implicit attitudes: participants with less positive attitudes towards the Southern US were more likely to compensate when the speaker was presented as Southern.
  • Vowel Matching: There was no main effect of speaker identity (i.e., no consistent exaggeration). However, there was a significant interaction with Working Memory. Participants with higher WM showed a greater effect of exaggeration when the speaker was presented as Southern, but this effect was absent when the speaker was presented as from Ohio. This suggests WM plays a role in integrating expectations at the phonetic level, but the expected exaggeration was not universally observed.

Limitations and Future Work

The authors acknowledge several points:

  • Unclear Phonetic Mechanism: While the interaction with WM suggests expectations influence phonetic processing, the lack of a main effect of identity means the predicted "exaggeration" effect wasn't consistently found. The authors suggest it's unclear why integration would be effortful at the phonetic level but not lexical.
  • Attitude Specificity: The effect of attitudes was restricted to the lexical task, and it's unclear why this is the case.
  • Future Directions: The authors emphasize the need for future research to further develop the proposed distinct mechanisms for phonetic and lexical processing. They suggest exploring the relative weight of each level of processing (phonetic vs. lexical) and how ambiguity influences this. They also strongly recommend that future studies avoid treating phonetic and lexical paradigms as equivalent, either choosing the most appropriate paradigm for a given question or comparing performance across both.

Summary updated

BibTeX
@inproceedings{levari2019different,
  author = {Shiri Lev-Ari and Robin Dodsworth and Jeff Mielke and Sharon Peperkamp},
  title = {{The Different Roles of Expectations in Phonetic and Lexical Processing}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure