Recognizing Speech in a Novel Accent: The Motor Theory of Speech Perception Reframed

Clément Moulin-Frier, Michael A. Arbib

Abstract

How do human listeners adapt to unfamiliar foreign accents with such remarkable speed? This paper, originally published in Biological Cybernetics, proposes a dual-stream neuro-computational framework to answer this question. By reframing the traditional Motor Theory of Speech Perception through a Hidden Markov Model (HMM) lens, the authors demonstrate how lexical feedback allows listeners to "bootstrap" recognition of novel accents using only a limited set of words.

Introduction: The Challenge of Novel Accents

For decades, the Motor Theory of Speech Perception has been a dominant paradigm, suggesting that we understand speech by internally simulating the motor gestures—such as tongue and lip movements—required to produce those sounds. However, this theory faces a significant challenge: how do we understand a speaker with a completely novel accent? If a listener has not yet learned the specific motor patterns of a foreign speaker, the simulation mechanism should, in theory, fail.

This research addresses the cognitive "Aha!" moment—the point where, after initial difficulty, a listener suddenly begins to decode a new accent with ease. The authors bridge the gap between auditory perception and motor simulation by proposing a statistical mapping process that facilitates rapid adaptation.

Methodology: A Computational Approach to Adaptation

The researchers developed a computational model based on Hidden Markov Models (HMMs) to simulate the listener's internal linguistic processes.

1. Representation and Modeling

Speech sounds (phones) were represented as feature vectors based on International Phonetic Association (IPA) standards, accounting for place of articulation, manner, and voicing. In the HMM framework:

  • Hidden States represent the listener’s native phonemes.
  • Observations represent the actual phonetic outputs produced by the foreign speaker.
  • Transitions represent the statistical probabilities of phoneme sequences within the native language.

2. The Adaptation Algorithm

The model begins in a "naïve" state. When it encounters a word it can identify through context or "lexical feedback," it employs the Viterbi algorithm to determine the most likely sequence of phonetic insertions, deletions, or substitutions used by the speaker. Following this, a Laplace succession rule is used to update the model's internal conversion table, refining the mapping between the speaker's unique acoustic realizations and the listener's native phonemic categories.

Key Insights: Reframing the Dual-Stream Architecture

The study offers several critical insights into the nature of speech perception:

  • Lexical Feedback as a Catalyst: Adaptation is not driven by isolated sounds but by recognizing known words to reverse-engineer the speaker's phonetic "code."
  • Dual-Stream Processing: The authors propose that speech perception is bifurcated. The Ventral Stream (Auditory-to-Meaning) handles standard, familiar speech. In contrast, the Dorsal Stream (Auditory-to-Motor) acts as a specialized mechanism that engages during high-effort conditions, such as processing heavy noise or learning a novel accent.
  • Mirror Neurons and Parity: Rather than being the primary engine of perception, the mirror neuron system is reframed as a tool for maintaining "parity"—a common ground between the listener's auditory input and their own motor output.

Results and Performance

The model's efficacy was tested using data from the Speech Accent Archive and the MRC Psycholinguistic Database.

  • Rapid Learning: In simulations involving a French-accented speaker, the model’s word recognition accuracy improved from a naïve 62% to 97% after processing only 35 words of lexical feedback.
  • Generalization: Once the model adapted to the small training set, it successfully generalized its findings to identify new, previously unheard words spoken in the same accent.
  • Statistical Significance: A two-way ANOVA confirmed that the learning process was significantly more impactful for foreign accents than for native speech, where the listener's internal model is already well-aligned.

Limitations and Future Directions

While the model provides a robust framework for understanding adaptation, several limitations remain. The current iteration relies on hand-coded phonetic transcriptions rather than raw acoustic signals, which contain significantly more variability. Additionally, the model treats speech as discrete units, overlooking the complexities of coarticulation where sounds overlap. Future research will focus on scaling this "bootstrapping" mechanism to larger vocabularies and validating the dual-stream shift through neuroimaging techniques like fMRI and EEG.

Summary updated

BibTeX
@inproceedings{moulinfrier2013recognizing,
  author = {Clément Moulin-Frier and Michael A. Arbib},
  title = {{Recognizing Speech in a Novel Accent: The Motor Theory of Speech Perception Reframed}},
  booktitle = {Biol. Cybern},
  year = {2013},
  url = {https://openreview.net/forum?id=eSBL1i7XSHO},
}
Figure