A comparative study of speech rate estimation techniques

Tomas Dekens, Mike Demol, Werner Verhelst, Piet VerhoeveInterspeech 2007

TL;DR

This paper tackles the challenge of accurately estimating speech rate across different languages and speaking styles. By comparing eight existing techniques on a novel multilingual dataset, the authors found that the "Temporal Correlation and Selected Sub-band Correlation" (tcssbc) method offers the most robust performance, particularly excelling at high speech rates where other methods falter.

What Problem Does This Paper Address?

Estimating how fast someone is speaking (speech rate) is crucial for many speech technology applications, like automatic speech recognition (ASR). If an ASR system knows the speaking rate, it can adjust its models to better understand the speech, leading to improved accuracy. However, a major hurdle in developing and comparing speech rate estimation techniques has been the lack of standardized evaluation. Researchers have historically tested their methods on different datasets, making it difficult to objectively determine which techniques are truly superior. This paper aims to bridge that gap by evaluating a range of established speech rate estimation methods on a single, diverse, multilingual test database.

Methods

The paper evaluates eight different speech rate estimation techniques. Most of these methods rely on detecting the number of vowels or diphthongs in a speech signal, as each syllable typically contains one such sound. The core idea is to count these vowel/diphthong occurrences per second to derive the speech rate.

The methods can be broadly categorized:

  • Spectral Moment-based: enrate estimates speech rate based on the first spectral moment of the broadband energy envelope. It's unique in not directly outputting syllables per second.
  • Peak Counting-based: Several methods (peakrate, mpeakrate, peakfrate) involve detecting peaks in the energy envelope of the speech signal or its sub-bands. These peaks are hypothesized to correspond to vowel onsets.
  • Combined Estimators: mrate combines enrate with two peak-counting estimators.
  • Correlation-based: tcssbc (Temporal Correlation and Selected Sub-band Correlation) is a more complex method that uses cross-correlation between sub-band energy envelopes after temporal smoothing. This approach is designed to be more robust to certain speech sounds.
  • Vowel Onset Detection-based: vorate and voprate specifically focus on detecting vowel onset points (VOPs) using techniques like subharmonic summation and linear prediction, respectively.

Datasets

The primary dataset used for evaluation is a multilingual test database consisting of 552 sound files. This database covers six European languages (Dutch, English, French, Italian, Romanian, and Spanish) and includes utterances recorded at slow, normal, and fast speech rates. For each language, speakers were asked to produce eight different utterances. The average duration of these utterances is 28 seconds, with approximately 115 syllables per utterance. The authors note that the number of syllables per utterance was known, allowing for the calculation of a reference speech rate. A subset of the Dutch data was used for optimizing the parameters of the estimators.

Key Insights and Ideas

  • Underestimation at High Rates is Common: A significant finding is that most estimators tend to underestimate the actual speech rate when speakers talk very fast. This is attributed to the merging of acoustic peaks (representing vowels/diphthongs) due to co-articulation and the influence of voiced consonants.
  • tcssbc Shows Superior Robustness: The tcssbc method, with its sub-band correlation approach, appears to be less affected by peak merging. This makes it a more reliable estimator, especially at higher speech rates.
  • Language Independence (Mostly): The performance of the estimators was found to be largely independent of the specific language, with performance degradation primarily linked to speech rate rather than linguistic differences.
  • Reference Rate Nuances: The paper highlights potential inaccuracies in the reference speech rate itself, particularly due to speakers adding or omitting syllables (e.g., "schwa" sounds or elisions) which are difficult to account for in automated calculations.

Key Results

  • High Correlations: All evaluated methods, except enrate, achieved high correlation coefficients with the reference speech rate, indicating they capture the general trend.
  • enrate's Drawbacks: enrate showed the lowest correlation and its output (in Hz, not syllables/sec) made direct comparison difficult, rendering it less suitable for this task.
  • Overestimation at Low Rates: vorate and voprate tended to overestimate speech rate at slow speaking speeds, likely due to misinterpreting other speech sounds as vowel onsets.
  • tcssbc Outperforms: The tcssbc method demonstrated the smallest errors (low Mean Squared Error) and high correlation, making it the most suitable technique among those tested. For instance, in the Dutch dataset, tcssbc achieved a correlation of 0.9331 and a mean error of -0.0186. In French, it achieved a correlation of 0.9242 and a mean error of -0.1884.

Limitations and Future Work

  • Reference Rate Accuracy: The authors acknowledge that the reference speech rate might have minor inaccuracies due to natural speech variations like added or omitted syllables, especially at extreme speaking rates.
  • Peak Merging: While tcssbc mitigates this, peak merging remains a fundamental challenge for many peak-counting based estimators at high speech rates.
  • Future Work: The paper implicitly suggests that further research could focus on refining peak detection algorithms to better handle co-articulation or exploring alternative acoustic features that are less susceptible to these effects. Investigating how these estimators perform on even more diverse languages or specific speech disorders could also be valuable.

Summary updated

BibTeX
@inproceedings{dekens2007comparative,
  author = {Tomas Dekens and Mike Demol and Werner Verhelst and Piet Verhoeve},
  title = {{A comparative study of speech rate estimation techniques}},
  booktitle = {Interspeech},
  year = {2007},
}

Loading related content...

Figure