Rhythm measures with language-independent segmentation

Anastassia Loukina, Greg Kochanski, Chilin Shih, Elinor Keane, Ian WatsonInterspeech 2009

TL;DR

This paper introduces a novel, language-independent method for segmenting speech into "vowel-like" and "consonant-like" regions, enabling consistent application of rhythm measures across diverse languages. While this approach improves robustness and allows for larger-scale analysis, it reveals that language rhythm is a complex, multi-dimensional phenomenon, and no single measure can fully capture it.

What Problem Does This Paper Address?

Understanding the rhythmic properties of spoken languages is crucial for fields like linguistics, speech processing, and even infant language acquisition. Traditionally, researchers have relied on manual segmentation of speech signals to identify vowel and consonant segments, which are then used to calculate various "rhythm measures" (RMs). However, this manual approach is subjective, time-consuming, and prone to inconsistencies, making it difficult to compare results across studies or languages. Furthermore, the reliance on phonological categories for segmentation might obscure acoustic differences that are actually key to how rhythm is perceived, especially by pre-lexical infants who can discriminate languages without explicit linguistic knowledge. This paper aims to overcome these limitations by developing an automatic, language-independent segmentation method and evaluating its effectiveness in capturing cross-linguistic rhythmic differences.

Methods

The core innovation of this paper lies in its automatic segmentation algorithm. Instead of relying on phonological labels, the algorithm segments speech based on two acoustic features: loudness and irregularity (or aperiodicity) of the waveform. This process categorizes speech segments into three types: silences, "vowel-like" segments (nearly periodic waveform), and "consonant-like" segments (non-periodic waveform, potentially including frication or rapid changes). This acoustic-based segmentation is designed to be consistent across languages and potentially closer to how pre-lexical infants perceive speech.

Once segmented, the paper applies 15 established rhythm measures to the data. These measures quantify various aspects of the timing and variability of vowel and consonant intervals. The authors also explore three different ways of handling pauses and pre-pausal syllables when calculating these measures, finding that omitting final syllable intervals generally yields slightly better results.

To evaluate the effectiveness of these measures, the researchers employ classifier techniques. They train classifiers to predict the language of origin based on one or more rhythm measures. The success rate of these classifiers serves as an indicator of how well the rhythm measures can distinguish between languages.

Datasets

The study utilizes a corpus of 1843 short texts recorded from 41 speakers of five languages: Southern British English (E), Standard Greek (G), Standard Russian (R), Standard French (F), and Taiwanese Mandarin (M). The texts were primarily extracts from "Harry Potter," fables, and the fairytale "Cinderella," either in their original language or in translation. Speakers were native monolinguals aged 20-28, recorded in a soundproof room at 16 kHz sampling rate.

Key Insights and Ideas

  • Acoustic Segmentation is Key: Language-independent acoustic segmentation (based on loudness and irregularity) is a viable and robust alternative to manual, phonologically-driven segmentation for rhythm analysis. It allows for consistent comparisons across languages and larger datasets.
  • Rhythm is Multi-Dimensional: No single rhythm measure, even when combined with speech rate, can reliably distinguish between all languages. This suggests that language rhythm is a complex phenomenon best described by at least two, and possibly more, dimensions.
  • Within-Language Variation is Significant: The variability in rhythm measures within a single language is substantial and comparable to the differences between languages. This makes it challenging to classify languages based on short speech samples.
  • Acoustic vs. Phonological Categories: The automatic segmentation highlights that acoustic realizations of phonological categories can differ significantly across languages, impacting rhythm measures. For instance, the classification of English [h] as "vowel-like" or part of silence, or devoiced vowels/sonorants as "consonant-like," challenges traditional phonetic classifications.
  • Speech Rate Matters, But Isn't Everything: While speech rate alone doesn't reliably distinguish languages, it plays a role in rhythm and can enhance the performance of certain rhythm measures, particularly those not normalized for rate.

Key Results

  • Classifiers based on single rhythm measures achieved success rates only slightly above chance (e.g., 43% for the best single measure, VnPVI, compared to a 30% chance level for 5 languages).
  • Measures based on vocalic intervals generally performed better than those based on consonantal variability.
  • Two-dimensional classifiers (combining two rhythm measures) improved performance, with the most efficient combinations reaching around 49-50% correct classification. However, no pair of measures correctly classified more than half of the data.
  • Higher-dimensional classifiers (15 or 45 measures) did not significantly improve the overall success rate beyond the best two-dimensional classifiers (around 50-52%).
  • The multidimensional classifier (using all 45 measures) showed less difference in correctly identified texts across languages compared to pairs of measures, indicating a more balanced representation of rhythmic features.
  • The automatic segmentation successfully captured known acoustic differences between languages, such as the realization of stop consonants in Mandarin and the voicing of plosives in French and English.

Limitations and Future Work

The authors acknowledge that the within-language variation is a significant limitation, making reliable language identification from short samples difficult. They also note that while higher-dimensional classifiers showed some improvement, the gains were marginal, suggesting that the current set of rhythm measures might be capturing only a limited number of underlying rhythmic dimensions.

Future work could explore:

  • Investigating more than two or three dimensions of rhythm, as the current measures might be highly correlated.
  • Developing new rhythm measures that capture different aspects of speech timing and variability.
  • Further exploring the interaction between speech rate and other rhythmic features.
  • Validating the language-independent acoustic segmentation with perceptual studies, especially concerning infant language acquisition.

Summary updated

BibTeX
@inproceedings{loukina2009rhythm,
  author = {Anastassia Loukina and Greg Kochanski and Chilin Shih and Elinor Keane and Ian Watson},
  title = {{Rhythm measures with language-independent segmentation}},
  booktitle = {Interspeech},
  year = {2009},
}

Loading related content...

Figure