Comparing and combining modeling techniques for sentence segmentation of spoken Czech using textual and prosodic information

Jáchym Kolář, Yang Liu2010 · DOI

Abstract

This paper deals with automatic sentence boundary detection in spoken Czech using both textual and prosodic information. This task is important to make automatic speech recognition (ASR) output more readable and easier for downstream language processing modules. We compare and combine three statistical models—hidden Markov model, maximum entropy, and adaptive boosting. We evaluate these methods on two Czech corpora, broadcast news and broadcast conversations, using both manual and ASR transcripts. Our results show that superior results are achieved when all the three models are combined via posterior probability interpolation, and that there is substantial difference among the three methods when using different knowledge sources, as well as in different genres. Feature analysis also reveals significant differences in prosodic feature usage patterns between the two genres.

Introduction

Automatic Speech Recognition (ASR) systems typically generate a continuous stream of words, often referred to as a "word salad," lacking punctuation and structural markers. This lack of structure poses a significant challenge for human readability and downstream natural language processing tasks such as machine translation or sentiment analysis. This research focuses on Sentence Unit (SU) segmentation for the Czech language, which presents unique linguistic challenges:

  • Morphology: As a highly inflectional language, Czech words change form frequently, resulting in a vast vocabulary of distinct word forms.
  • Syntax: Flexible word order in Czech contrasts with the more rigid structures of languages like English.
  • Prosody: Unlike English, which uses "preboundary lengthening" as a primary cue, Czech utilizes vowel length for lexical distinction, making acoustic boundary detection more subtle and reliant on pitch movements.

Methodology

The study evaluates and integrates three distinct statistical modeling techniques:

  1. Hidden Markov Model (HMM): A generative model that treats sentence boundaries as hidden states, combining a language model with a prosody model at the score level.
  2. Maximum Entropy (MaxEnt): A discriminative model effective at handling overlapping features. Prosodic scores were discretized into binary features to accommodate the model's requirements.
  3. Adaptive Boosting (Boosting): Utilizing the ICSIboost implementation, this model combines "weak learners" (decision stumps) into a robust classifier, handling raw prosodic features directly.

Feature Engineering

  • Textual Features: Included word identities, part-of-speech (POS) tags, and Automatically Induced Classes (AICs) to mitigate data sparsity.
  • Prosodic Features: Extracted features related to pauses, duration (vowels and rhymes), pitch (F0) slopes, and energy (RMS) levels.

Experimental Setup

The models were tested on two primary Czech corpora annotated according to the LDC’s Metadata Extraction (MDE) standard:

  • Broadcast News (BN): 174.8k words of formal, mostly read speech.
  • Broadcast Conversations (BC): 159.1k words of spontaneous, informal dialogue.
  • Auxiliary Data: A 107-million-word corpus of Czech broadcast transcripts was used to enhance the language models.

Evaluation was conducted on both human-corrected "Reference" transcripts and raw ASR output, which exhibited word error rates of approximately 12% for BN and 29% for BC.

Results and Discussion

The primary evaluation metric is the Boundary Error Rate (BER), representing the percentage of incorrectly identified or missed boundaries.

Comparison of individual model performance (HMM, MaxEnt, Boosting) across different test conditions.

The analysis revealed that no single model is universally superior. HMMs proved robust for spontaneous speech, while Boosting excelled when both textual and prosodic cues were available. MaxEnt performed relatively poorly on Broadcast News, likely due to the discretization of nuanced acoustic information. The best performance was consistently achieved by combining all three models through posterior probability interpolation.

Detailed Boundary Error Rate (BER) results for individual models and their combination.

Key Insights

  • Genre-Specific Prosody: Broadcast News anchors utilize distinct pitch falls to signal sentence endings, whereas these cues are less reliable in spontaneous conversations (BC).
  • Language-Specific Constraints: Czech speakers rely more on pitch movements than lengthening for boundary signaling, confirming the impact of lexical vowel length on prosodic patterns.
  • Feature Complexity: Optimal performance required 11 prosodic features for BN but 17 for BC, reflecting the increased complexity of informal speech.

Conclusion

The ensemble approach significantly outperformed individual models, maintaining a BER under 4% even in challenging spontaneous speech ASR conditions. Future work will explore more sophisticated methods for integrating prosodic information into discriminative models to further bridge the performance gap between formal and informal speech genres.

Summary updated

Abstract

This paper deals with automatic sentence boundary detection in spoken Czech using both textual and prosodic information. This task is important to make automatic speech recognition (ASR) output more readable and easier for downstream language processing modules. We compare and combine three statistical models -hidden Markov model, maximum entropy, and adaptive boosting. We evaluate these methods on two Czech corpora, broadcast news and broadcast conversations, using both manual and ASR transcripts. Our results show that superior results are achieved when all the three models are combined via posterior probability interpolation, and that there is substantial difference among the three methods when using different knowledge sources, as well as in different genres. Feature analysis also reveals significant differences in prosodic feature usage patterns between the two genres.
BibTeX
@misc{kolar2010comparing,
  author = {Jáchym Kolář and Yang Liu},
  title = {{Comparing and combining modeling techniques for sentence segmentation of spoken Czech using textual and prosodic information}},
  year = {2010},
  doi = {10.21437/speechprosody.2010-76},
}
Figure