Synthetic correction of deviant speech – children's perception of phonologically modified recordings of their own speech

Sofia StrömbergssonInterspeech 2012

TL;DR

This study investigates how children with phonological impairments (PI) perceive synthetically corrected versions of their own speech. The findings suggest that while these children struggle to identify errors in their own recorded speech, they are generally accepting of phonologically accurate synthetic modifications, indicating potential for therapeutic applications.

What Problem Does This Paper Address?

Children with phonological impairments (PI) often exhibit not only difficulties in producing speech sounds correctly but also in perceiving these errors, especially when listening to their own voices. This discrepancy between external (others' speech) and internal (own speech) perception is a significant challenge in speech therapy. Current methods for providing feedback often rely on manual editing, which is time-consuming and lacks the immediacy needed for effective intervention. This paper explores the potential of automated synthetic speech modification to provide children with PI with a more objective and accessible way to become aware of their speech production errors. Specifically, it addresses whether children with PI can detect synthetic modifications of their own speech and how their perception differs from children with typical speech development.

Methods

The core of this research lies in a novel method for synthetic speech modification. This technique uses a unit selection approach to replace the initial voiceless plosive in a recorded speech sample with a plosive from another recording. This allows for two types of modifications:

  1. Phonological identity change: Replacing a sound with a different one (e.g., /t/ with /k/).
  2. Subtle modification: Replacing a sound with a phonetically similar version of the same sound, or a version from another child's recording of the same sound.

The study involved several experimental tasks designed to assess children's perception:

  • External Word Identification (EXTIDENT): Children judged the phonological accuracy of recordings from other children (both original and modified).
  • Internal Modified Identification (INTMOD1 & INTMOD2): Children judged the phonological accuracy of their own recorded speech, immediately after production (INTMOD1) and after a delay (INTMOD2). These recordings were either original, synthetically corrected, or synthetically modified without changing the phonological identity.
  • Internal Speaker Judgment (INTSPKR): Children judged whether a recording was their own voice or not, after synthetic modifications.
  • Internal Word Identification (INTIDENT1 & INTIDENT2): Children judged the phonological accuracy of their own speech production in real-time (INTIDENT1) and from recordings (INTIDENT2). These tasks were only performed by children with PI.

Datasets

The study utilized recordings of children's speech. No new datasets were introduced; instead, existing recordings were modified using the described synthetic speech technique. The evaluation was conducted with two groups of children:

  • Children with Phonological Impairment (PI): 8 children (aged 4;0 to 7;2) with specific patterns of velar fronting or dental backing.
  • Children with Typical Speech and Language (nPI): 20 children (aged 4;0 to 6;3) serving as a control group.

Key Insights and Ideas

  • Internal vs. External Perception Divide: A crucial finding is that children with PI perform comparably to typically developing children when judging others' speech. However, when judging their own recorded speech, their performance significantly lags behind. This reinforces the idea that the primary challenge for children with PI lies in self-monitoring and internal perception, not external auditory discrimination.
  • Subtle Modifications Go Unnoticed: The study strongly suggests that sub-phonemic modifications (e.g., slight variations in the production of the same sound) are too subtle for both groups of children to detect. This is a key finding for the practical application of synthetic correction, as it implies that children might accept phonologically correct versions of their speech without necessarily noticing the artificiality, which could be beneficial in therapy.
  • Acceptance of Corrected Speech: When synthetic modifications result in phonologically correct speech, children generally accept it as accurate, even if it's not their original production. This indicates that the synthetic correction method is effective in producing perceptually acceptable speech.

Key Results

  • Significant Group Differences in Internal Perception: In tasks requiring children to judge the phonological accuracy of their own recorded speech (INTMOD1 and INTMOD2), children with PI performed significantly worse than the typically developing children (nPI).
  • No Significant Differences in External Perception: In the task where children judged the phonological accuracy of other children's speech (EXTIDENT), there was no significant difference in performance between the PI and nPI groups.
  • Speaker Identity Judgment: Children with PI also showed a significant difference in judging speaker identity in their own modified recordings compared to the nPI group, suggesting they are less adept at recognizing their own voice after modifications.
  • Subtle Modifications are Undetectable: For both groups, sub-phonemic modifications were too subtle to be detected, meaning children often mistook modified stimuli for their own original productions.

Limitations and Future Work

  • Limited Sample Size for PI Group: The study included only 8 children with PI, and the authors suggest that a larger sample size might reveal more significant results, particularly regarding the discrepancy between online and delayed self-judgment of speech production.
  • Intent vs. Product Hypothesis: While the study aimed to test the hypothesis that children with deviant speech judge their intent rather than the product of their speech, the results were inconclusive due to large individual variations. Further research with more participants is needed to explore this.
  • Clinical Applicability: The authors highlight the encouraging results regarding the acceptance of synthetically corrected speech, suggesting that this technology holds promise for clinical intervention. Future work could focus on developing and evaluating therapeutic protocols that leverage this synthetic modification technique.

Summary updated

BibTeX
@inproceedings{strombergsson2012synthetic,
  author = {Sofia Strömbergsson},
  title = {{Synthetic correction of deviant speech – children's perception of phonologically modified recordings of their own speech}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure