Analysis of context-dependent segmental duration for automatic speech recognition

Xue Wang, L.c.w. Pols, Louis F. M. ten Bosch4th 1996 · DOI

TL;DR

This paper investigates how the duration of speech sounds (phones) changes based on their surrounding context, using the TIMIT database. By analyzing these context-dependent durations, the authors aim to improve automatic speech recognition (ASR) systems, suggesting that incorporating this knowledge into the post-processing stage can lead to better performance.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems, particularly those based on Hidden Markov Models (HMMs) and Artificial Neural Networks (ANNs), have achieved impressive performance. However, they often struggle with the inherent variability in human speech. While statistical models capture general patterns, they can miss subtle, yet predictable, variations. For instance, stressed vowels tend to be longer and have different spectral qualities than unstressed ones. This paper focuses on one such predictable variation: the duration of speech segments (phones). Understanding and modeling how phone durations change based on their linguistic and phonetic context (e.g., surrounding sounds, word stress, speaking rate) could provide valuable information to enhance ASR systems beyond what purely statistical models can achieve.

Methods

The authors employ two primary analytical approaches:

  1. Duration Distribution Analysis: They examine the distribution of phone durations under the influence of various contextual factors. This involves plotting histograms and analyzing how the shape and parameters of these distributions change when specific factors are considered.
  2. Hierarchical Analysis of Variance (ANOVA): To quantitatively assess the contribution of different factors to duration variation, they developed a hierarchical ANOVA. This method allows them to study the numerical impact of 11 identified contextual factors on phone duration. The paper notes that a standard ANOVA is challenging due to the complexity of the factorial design, data size, nested factors, and other statistical issues.

Datasets

The primary dataset used is the TIMIT database. Specifically, the authors utilized the hand-segmented TIMIT training set, comprising 3,696 utterances (si and sx sentences) from 462 speakers (326 male, 136 female). This dataset was chosen because it provides continuous speech from multiple speakers, which is closer to real-world scenarios, and crucially, it is fully hand-labeled, making it suitable for detailed phonetic analysis.

Key Insights and Ideas

  • Context is King for Duration: The study confirms that phone durations are not fixed but are significantly influenced by their surrounding linguistic and phonetic environment.
  • Predictable Variations Exist: Despite the inherent variability in speech, several contextual factors show consistent effects on phone duration. For example, vowel length, stress, and position within an utterance demonstrably alter duration.
  • Post-Processing Potential: The authors suggest that this context-dependent duration knowledge, which is difficult to fully integrate into the core HMM/ANN models, can be effectively utilized in the post-processing stage of an ASR system.
  • TIMIT's Complexity: The multi-speaker nature of TIMIT, while realistic, introduces more variability than single-speaker databases, making some effects less pronounced and requiring careful statistical analysis.
  • Speaking Rate Matters: Speaking rate has a near-linear relationship with absolute phone duration, highlighting its importance for ASR.

Key Results

  • Vowel Duration Effects:
    • Longer vowels are consistently longer than short vowels.
  • Vowels in stressed syllables are significantly longer than those in unstressed syllables.
  • Utterance-final vowels tend to be longer (utterance-final lengthening).
  • The effect of post-vocalic plosives on vowel duration was less pronounced in TIMIT compared to findings from single-speaker studies, suggesting increased variability in multi-speaker data.
  • ANOVA Contributions: The ANOVA revealed that factors like phone identity and phonetic class contribute significantly to duration variation. However, the paper notes that some factors that appeared consistent in distribution analysis (like stress) showed smaller contributions in the overall ANOVA, possibly due to interaction effects not fully captured.
  • Hand-Labeling vs. Viterbi: The study acknowledges a significant discrepancy (up to 20ms deviation for about 15% of phones) between hand-labeled durations and those obtained from automatic segmentation (Viterbi search), indicating a need for methods to handle these deviations.

Limitations and Future Work

  • Interaction Effects: The hierarchical ANOVA, as presented, primarily analyzes main effects. The authors note that interaction effects (e.g., how stress affects duration differently for long vs. short vowels) are crucial but not fully captured by this specific ANOVA structure.
  • Handling Deviations: The significant differences between hand-labeled and automatically segmented durations present a challenge. Future work should focus on how to incorporate this knowledge, perhaps by developing duration models that account for these deviations or by integrating them into the post-processing phase.
  • Broader Contextual Factors: While 11 factors were analyzed, the authors suggest that further research into other contextual factors, such as pitch, could also yield valuable insights for ASR.
  • Integration into ASR: The paper proposes integrating this duration knowledge into the post-processing part of the recognizer, likely through N-best sentence alternatives, as a practical next step. A more extended version of this paper is planned for publication.

Summary updated

BibTeX
@inproceedings{wang1996analysis,
  author = {Xue Wang and L.C.W. Pols and Louis F. M. ten Bosch},
  title = {{Analysis of context-dependent segmental duration for automatic speech recognition}},
  booktitle = {4th},
  year = {1996},
  doi = {10.21437/icslp.1996-305},
}
Figure