Important and new features with analysis for disfluency interruption point (IP) detection in spontaneous Mandarin speech

Che-Kuang Lin, Shu-Chuan Tseng, Lin-Shan LeeDiSS 2005

TL;DR

This paper introduces novel acoustic-prosodic features for detecting disfluency interruption points (IPs) in spontaneous Mandarin speech, specifically addressing the unique characteristics of the language. By analyzing the impact of individual features, they identify which ones are most crucial for different types of disfluencies, offering a more nuanced approach to speech processing.

What Problem Does This Paper Address?

Spontaneous human speech is inherently messy, filled with hesitations, repetitions, and self-corrections – collectively known as disfluencies. While standard speech recognition systems perform well on clean, well-formed speech, they struggle with these disfluencies, leading to errors and a lack of structural understanding. Accurately identifying the precise point where a disfluency begins (the interruption point or IP) is crucial for improving speech recognition accuracy and gaining deeper insights into conversational structure. This paper focuses on tackling this challenge specifically for Mandarin Chinese, a tonal language with a monosyllabic structure, which presents unique prosodic characteristics not always captured by models developed for other languages.

Methods

The researchers developed a comprehensive set of new acoustic-prosodic features, categorized into duration-related and pitch-related features. They then employed a maximum entropy model combined with a decision tree for IP detection. A key aspect of their methodology was a detailed feature analysis: they systematically removed individual features to assess their impact on detection performance (measured by recall rate degradation). This allowed them to pinpoint the most influential features for different disfluency types. To handle the imbalance between disfluency and non-disfluency instances, they used an ensemble sampling technique to equalize prior probabilities during training.

Datasets

The experiments were conducted on the Mandarin Conversational Dialogue Corpus (MCDC), a collection of 30 digitized spontaneous Mandarin conversations totaling 27 hours. For this study, 8 hours of annotated dialogues, produced by 16 speakers, were used. These dialogues were annotated for various spontaneous speech phenomena, including disfluencies.

Key Insights and Ideas

  • Mandarin-Specific Features Matter: The unique tonal nature and monosyllabic structure of Mandarin necessitate specialized features, as standard approaches might miss crucial prosodic cues.
  • Pitch is Key for Repairs: Pitch-related features, particularly those related to pitch level and range, are highly important for detecting overt repairs and partial repetitions. This aligns with the idea that repairs often involve re-establishing a natural intonation unit.
  • Duration for Repetitions: Duration-related features, especially those that jointly consider syllable and pause durations, are more critical for identifying direct repetitions. This suggests that changes in speaking rate are a strong indicator of repeated segments.
  • Abandoned Utterances Need Both: Detecting IPs in abandoned utterances benefits equally from both pitch and duration features, indicating a more complex disruption of speech flow.
  • Feature Importance Varies by Disfluency Type: There isn't a one-size-fits-all set of "most important" features. The analysis reveals a clear distinction in feature influence depending on whether the disfluency is a repetition, repair, or abandonment.

Key Results

The paper reports recall and precision rates for IP detection. The maximum entropy model achieved a recall of 56.38% and a precision of 81.95%, which the authors deem more suitable for recognition systems where false alarms are more detrimental than omissions. The detailed feature analysis revealed specific features that, when removed, caused significant recall degradation for different disfluency types. For instance, the removal of feature (a) (difference of pitch slope across boundary) caused a -17.25% recall degradation for abandoned utterances and overt repairs. Similarly, feature (b) (average pitch value within a syllable) was crucial for partial repetitions, with a -18.21% degradation.

Limitations and Future Work

The authors acknowledge that false alarms remain a significant problem for practical applications. They suggest that a promising direction for future work is to develop disfluency type-specific rule-based methods to further refine the discrimination of false alarms, thereby improving the overall precision of the detection system.

Summary updated

BibTeX
@inproceedings{lin2005important,
  author = {Che-Kuang Lin and Shu-Chuan Tseng and Lin-Shan Lee},
  title = {{Important and new features with analysis for disfluency interruption point (IP) detection in spontaneous Mandarin speech}},
  booktitle = {DiSS},
  year = {2005},
}
Figure