Acoustic Characteristics of Lexical Tone Disruption in Mandarin Speakers After Brain Damage

Wenjun Chen, Jeroen van de Weijer, Shuangshuang Zhu, Qian Qian, Manna WangInterspeech 2019

TL;DR

This study investigates how brain damage affects the production of lexical tones in Mandarin speakers. By analyzing acoustic features of tones produced by both healthy individuals and patients with left or right hemisphere damage, the researchers found that even "correctly" identified tones from brain-damaged speakers exhibit subtle acoustic differences from those of healthy speakers, suggesting a deeper impact on motor control than previously understood.

What Problem Does This Paper Address?

Mandarin is a tonal language where the pitch contour of a syllable distinguishes word meaning. While it's generally accepted that brain damage can disrupt tone production, there's been ongoing debate about where in the brain tone processing primarily occurs and how specific types of brain damage (left vs. right hemisphere) impact tone production. Previous research has often focused on left hemisphere damage (LHD), linking it to tone disruption due to its role in processing rapidly changing acoustic cues. However, some studies suggest tone production might be less lateralized, with both LHD and right hemisphere damage (RHD) groups showing impairments. This paper aims to clarify these patterns by performing detailed acoustic analyses on tones produced by Mandarin speakers with both LHD and RHD, comparing them to healthy controls, and investigating whether there are regular acoustic patterns in tone disruption.

Methods

The researchers recruited eight Mandarin speakers with brain damage (four LHD, four RHD) and four healthy control speakers (spouses of patients). They used a set of 24 monosyllabic words, each representing one of the four Mandarin tones, presented visually via colored pictures. Participants produced these words in isolation. The produced tones were then rated by ten native Mandarin speakers for correctness. Productions were classified into three groups: "Healthy speakers correct productions" (HC), "Patients' Correct productions" (PC), and "patients' incorrect productions" (PI).

For acoustic analysis, the fundamental frequency (F0) contour of the rime (vowel portion) of each syllable was extracted. Seven acoustic parameters were measured:

  • Pitch shift: Measures the flatness of the F0 contour for T1 (level tone).
  • Height of mean F0 & Height of min F0: Measure the overall pitch level relative to the speaker's mean F0, important for T3 (dipping tone).
  • Directional excursion: Measures the positive and negative span of the F0 contour, relevant for dynamic tones (T2, T4).
  • Slope: Measures the steepness of the F0 contour.
  • Timing of max F0 & Timing of min F0: Indicate when the peak or trough of the F0 contour occurs within the syllable.

Statistical analysis, specifically the Mann-Whitney-U test, was used to compare these parameters across the HC, PC, and PI groups due to violations of normality and homogeneity of variance.

Datasets

The study used a custom dataset of 24 monosyllabic Mandarin words, selected based on a word familiarity test for aphasic patients. These words were presented to participants via colored pictures. The produced tones were then categorized into HC, PC, and PI groups based on listener judgments. No new benchmark datasets were introduced; rather, existing listener judgment methods were employed to categorize productions for subsequent acoustic analysis.

Key Insights and Ideas

  • Subtle acoustic differences even in "correct" tones: Even tones produced by brain-damaged patients that were identified as "correct" by native listeners showed acoustically significant differences from healthy speakers' productions. This suggests that brain damage impacts the fine motor control of tone production more broadly than just producing outright errors.
  • No clear lateralization of impairment: Both LHD and RHD patients showed disruptions in tone production, and there was no significant difference in the number of correct or incorrect productions between the two groups. This challenges the notion that tone production is solely or primarily a left-hemisphere function.
  • Dynamic tones are more vulnerable: The study found a pattern of tone susceptibility, with the dipping tone (T3) being the most vulnerable to disruption, followed by the falling tone (T4), then the rising tone (T2), and finally the level tone (T1) being the most resistant. This aligns with the idea that more complex dynamic pitch changes require finer motor control.
  • PI productions are more disrupted than PC: Incorrect productions (PI) were found to be even more acoustically deviant from healthy speakers than the patients' own "correct" productions (PC), highlighting a spectrum of disruption.

Key Results

  • T1: Patients' correct T1 productions were less level than healthy speakers', showing greater pitch shift. Incorrect T1 productions exhibited even larger pitch fluctuations and lower minimum F0 height.
  • T2: Patients' correct T2 productions had a smaller excursion range and reached their maximum F0 earlier than healthy speakers. Incorrect T2 productions showed even smaller excursions, sometimes negative, and reached their minimum F0 earlier.
  • T3: Patients' correct T3 productions were significantly lower in mean and minimum F0 height compared to healthy speakers. Both PC and PI T3 productions were less lowered and less dipped, often showing leveling contours.
  • T4: Patients' correct T4 productions were less ranged and steep, with maximum F0 reached later than in healthy speakers. Incorrect T4 productions showed more variable excursions and reduced steepness.
  • Listener judgments: T1 was the most accurately identified tone (74% correct), while T3 was the least (18% correct). This supports the observed acoustic findings.

Limitations and Future Work

The authors acknowledge that the sample size of brain-damaged patients in this study is comparatively small. They suggest that future research should focus on how disruption degrees vary across different types of brain damage and potentially investigate a larger sample size to draw more robust conclusions. Further research is also needed to explore the proposed link between impaired motor control and the difficulty in producing dynamic tones.

Summary updated

BibTeX
@inproceedings{chen2019acoustic,
  author = {Wenjun Chen and Jeroen van de Weijer and Shuangshuang Zhu and Qian Qian and Manna Wang},
  title = {{Acoustic Characteristics of Lexical Tone Disruption in Mandarin Speakers After Brain Damage}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure