A discriminative analysis within and across voiced and unvoiced consonants in neutral and whispered speech in multiple indian languages

G. Nisha Meenakshi, Prasanta Kumar GhoshInterspeech 2015

TL;DR

This paper investigates how well voiced and unvoiced consonants can be distinguished in whispered speech compared to normal speech, across several Indian languages. They found that while discrimination generally decreases in whispers, the degree of loss varies significantly by consonant and language, suggesting that articulatory adjustments in whispering help preserve consonant identity.

What Problem Does This Paper Address?

Distinguishing between voiced and unvoiced consonants is fundamental to speech perception and processing. In normal speech, this distinction is primarily cued by vocal fold vibration (voicing). However, whispered speech, by definition, lacks vocal fold vibration, making this distinction much more challenging. This poses a significant problem for speech recognition systems and other speech processing technologies that rely on these phonetic cues, especially when dealing with whispered speech, which is common in noisy environments or for discreet communication. This paper aims to quantify the extent to which this crucial phonetic distinction is preserved or degraded in whispered speech across multiple Indian languages, using both acoustic analysis and human perception tests.

Methods

The researchers employed a two-pronged approach: objective acoustic analysis and subjective listening tests.

For the objective analysis, they used Mel-frequency cepstral coefficients (MFCCs) to capture vocal tract information. They computed the acoustic distance between consonant pairs using a 36-dimensional feature vector (excluding energy coefficients to balance whisper and neutral speech intensity) and aligned sequences using Dynamic Time Warping. They analyzed two types of acoustic distances:

  1. Intra-consonant distance: Measuring the variability within repetitions of the same consonant in neutral (DN) and whispered (DW) speech. This helps understand the acoustic space occupied by each consonant.
  2. Inter-consonant (V-UV pair) distance: Measuring the acoustic difference between a voiced consonant and its unvoiced counterpart (DV-UV and DW-UV). This directly quantifies the discriminability between voiced and unvoiced pairs.

For the subjective analysis, they conducted listening tests where native speakers of six Indian languages identified consonant pairs in VCV (vowel-consonant-vowel) stimuli. Listeners were presented with voiced and unvoiced consonant pairs and asked to choose the correct identity, providing a confidence score. This allowed them to calculate human recognition accuracy for individual consonants and V-UV pairs.

Datasets

The study utilized VCV (vowel-consonant-vowel) stimuli recorded from 12 native speakers (one male and one female for each of the six languages: Kannada, Telugu, Bengali, Hindi, Malayalam, and Tamil). The phonemic transcriptions of the VCV stimuli included seven voiced-unvoiced consonant pairs spanning various places of articulation: /b/-/p/, /d/-/t/, /g/-/k/, /v/-/f/, /z/-/s/, /tf/-/S/, and /dʒ/-/tʃ/. A total of 1174 neutral VCV tokens and 1141 whispered VCV tokens were collected. For the listening test, six native speakers (different from the recording subjects) served as listeners.

Key Insights and Ideas

  • Whispering shrinks the acoustic space: The intra-consonant acoustic distance (variability) generally decreased in whispered speech compared to neutral speech for most consonants, suggesting a more constrained acoustic space.
  • V-UV discrimination is reduced, but not eliminated: Acoustic analysis showed a decrease in the discriminative distance between voiced and unvoiced consonant pairs in whispered speech across all languages. However, human listeners could still distinguish many pairs with reasonable accuracy.
  • Consonant-specific and language-specific variations are significant: The degree of V-UV discrimination loss varied greatly depending on the specific consonant pair (e.g., /g/-/k/ showed a large reduction, while /z/-/s/ showed a small one acoustically) and the language.
  • Acoustic distance doesn't perfectly predict perceptual confusion: A reduction in acoustic distance between a V-UV pair did not always translate to lower human recognition accuracy, and vice-versa. This highlights the complex interplay of acoustic cues and perceptual strategies.
  • Articulatory hyper-articulation might compensate: The paper suggests that exaggerated articulatory movements in whispered speech, as reported in prior work, may help maintain consonant identity despite the absence of voicing.

Key Results

  • Acoustic Discrimination Loss: The V-UV consonant discrimination acoustically decreased in whispered speech, with the largest reduction observed for the /g/-/k/ pair (22.75%) and the smallest for the /z/-/s/ pair (3.35%).
  • Human Recognition Accuracy (Neutral): In neutral speech, listeners achieved high recognition accuracy (average 97.92%) with high confidence scores for individual consonants.
  • Human Recognition Accuracy (Whispered): Average recognition accuracy dropped to 69.58% in whispered speech, with lower confidence scores, indicating increased perceptual difficulty.
  • Whispered V-UV Pair Performance: While some V-UV pairs showed significant drops in recognition accuracy (e.g., /d/-/tf/ pair with 44.43% reduction), many still maintained over 60% accuracy.
  • Language Performance: Whispered Tamil VCVs had the highest subjective V-UV classification accuracy (85.71%), while Telugu had the lowest (58.93%), demonstrating significant cross-linguistic variation.
  • Specific Consonant Preservation: The retroflex stop consonant /d/ and the labiodental fricative /v/ (in languages where it exists) were found to preserve their identity to a greater extent than other voiced consonants in whispered speech.

Limitations and Future Work

  • Limited Stimuli: The study used VCV stimuli, which are simplified and may not fully represent the complexities of spontaneous whispered speech.
  • Consonant-Specific Hyper-articulation: While mentioned, the direct link between specific articulatory adjustments and perceptual outcomes was not exhaustively explored.
  • Future Work: The authors propose using these findings to develop automatic V-UV consonant classification systems for both read and spontaneous whispered speech. They also aim to investigate the role of articulatory movements more deeply.

Summary updated

BibTeX
@inproceedings{meenakshi2015discriminative,
  author = {G. Nisha Meenakshi and Prasanta Kumar Ghosh},
  title = {{A discriminative analysis within and across voiced and unvoiced consonants in neutral and whispered speech in multiple indian languages}},
  booktitle = {Interspeech},
  year = {2015},
}
Figure