Time-Domain, Digital Segmentation Of Connected Natural Speech

TL;DR

This paper presents a novel time-domain digital segmentation algorithm for connected natural speech, aiming to precisely locate phonemes and phoneme-like elements. The method achieves this by first separating pauses and voiced/voiceless sections, then analyzing vocal tract dynamics to identify stationary and dynamic segments, ultimately leading to a more robust speech recognition system.

What Problem Does This Paper Address?

Automatic speech recognition (ASR) systems face a fundamental challenge: the immense redundancy in acoustic speech signals compared to their written counterparts. Classifiers often struggle with this high information content, necessitating a preprocessor to extract significant, yet reduced, parameters that retain phonetic information. Furthermore, recognizing continuous speech requires discrete output units, and while words or syllables are too coarse, phonemes or similar elementary units demand precise temporal localization. This paper tackles the problem of speech segmentation, specifically the accurate time localization of phonemes within continuous speech, as a crucial preprocessing step for ASR.

Methods

The proposed segmentation algorithm employs a multi-stage approach:

  • Pre-segmentation: This initial step distinguishes between pauses, voiced, and voiceless sections of the speech signal. It utilizes a level distribution histogram to robustly discriminate pauses from speech, even with varying noise levels. Voiced and voiceless sections are differentiated using the ratio of the level of the differenced signal ($L_D$) to the level of the original signal ($L_S$).
  • Main Segmentation (Dynamic vs. Stationary): The core of the algorithm focuses on identifying "stationary" segments (where vocal tract position changes slowly) and "dynamic" segments (where vocal tract position changes rapidly). This is based on the assumption that phoneme realization involves a dynamic start, a stationary middle, and a dynamic end.
  • Parameter Extraction: Key parameters like $L_S$, $L_D$, and the ratio $Q_S$ are measured pitch-synchronously (or at fixed intervals if pitch is unavailable).
  • Segment Length Function (SLF): A novel "segment-length-function" (SLF) is introduced. This function, derived from the relative change of parameters between microsegments, helps define "minimal segments" by identifying points of significant parameter change. The SLF is designed to emphasize dynamic segments.
  • Primary Segmentation: Minimal segments are then classified as "dynamic," "stationary," or "undefined" based on the SLF and parameter behavior. Adjacent segments with the same classification are grouped into "primary" segments.
  • Transitional Segments: Dynamic segments are further refined into "transitional" segments. This involves analyzing the direction and monotony of parameter changes within dynamic segments. Monotonic segments are labeled "transitional-rising" or "transitional-falling." Non-monotonic dynamic segments are subdivided into monotonic fractions.
  • Undefined Segment Handling: A significant portion of segments are initially classified as "undefined." A dedicated algorithm eliminates these by considering the classifications of their neighboring segments and the direction characteristics. Ambiguous cases are labeled "may-be-transitional" or "may-be-stationary."
  • Corrections: The algorithm includes correction steps to handle parameter errors (e.g., from pitch detection failures) and to identify "hidden stationary segments" that might be missed due to the algorithm's emphasis on dynamic segments.

Datasets

The paper mentions that the segmentation procedure was applied to speech signals consisting of a list of 200 isolated German words, uttered by seven speakers. It also refers to results for a whole sentence and an utterance from a previous work ("Do any samples contain tridymite?"). No new benchmarks or datasets appear to be introduced.

Key Insights and Ideas

  • Emphasis on Dynamic Segments: The paper highlights that dynamic segments are more crucial for speech perception than stationary ones. This insight guides the algorithm's design, particularly in the definition of the SLF and minimal segments.
  • Novel Segment Length Function (SLF): The introduction of the SLF as a measure of segment length based on parameter changes is a key innovation. It provides a robust way to identify points of significant acoustic events.
  • Robust Pause/Speech Discrimination: Using a level distribution histogram for pause-speech discrimination is a significant improvement over fixed thresholds, making the system more resilient to varying noise conditions.
  • Handling of "Undefined" Segments: The systematic approach to classifying and eliminating "undefined" segments by considering neighboring context is crucial for achieving a complete segmentation.
  • Parameter Sensitivity and Correction: The acknowledgment that segmentation parameters can be sensitive to signal level and the inclusion of correction mechanisms for parameter errors and hidden segments demonstrate a practical approach to building a reliable system.

Key Results

  • The algorithm achieved promising results, with an error rate of 1.3% for missed phoneme boundaries and 0.9% for missed stationary segments in isolated German words.
  • The pause/speech discrimination contributed 0.8% to the total error score, and voiceless/voiced discrimination contributed 0.3%.
  • An additional 2.1% of boundaries were inserted, incorrectly dividing one phoneme into multiple stationary segments.
  • The paper demonstrates the algorithm's performance on a German word ("Belobigung") and an English utterance ("Do any samples contain tridymite?"), showing the segmentation results for various parameters and the final segmentation.

Limitations and Future Work

  • Speaker Variability: The authors acknowledge that inaccuracies introduced by speakers (e.g., additional spoken phonemes, phoneme reduction, erroneous pronunciation) limit the overall performance of any segmentation procedure.
  • Short Glides and Diphthongs: The algorithm sometimes misses short glides and diphthongs, and may require admitting several allophones for certain phonemes (like /r/).
  • Ambiguous Segment Combinations: Certain combinations of segments (e.g., undefined segments between stationary or transitional ones) can lead to ambiguous results, requiring careful handling.
  • Parameter Sensitivity: The SLF's sensitivity to the absolute value of the signal level at phoneme boundaries is noted, suggesting the need for careful selection of correction factors and thresholds.
  • Future Work: The paper implies that further refinement of the parameter selection and thresholding could improve accuracy. The presented algorithm is a component of a larger recognition system, suggesting that its integration and performance within that system would be a natural next step.

Summary updated

BibTeX
@inproceedings{1975time,
  title = {{Time-Domain, Digital Segmentation Of Connected Natural Speech}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1975},
}
Figure