Acoustic-phonetic features for refining the explicit speech segmentation

Antonio Marcos Selmini, Fábio ViolaroInterspeech 2007

TL;DR

This paper presents a novel method to improve the accuracy of automatic speech segmentation into phones by incorporating acoustic-phonetic features. By refining boundaries initially identified by Hidden Markov Models (HMMs), the system significantly reduces segmentation errors, bringing them closer to human expert performance.

What Problem Does This Paper Address?

Accurate speech segmentation, the process of dividing an audio recording into meaningful units like phones, is crucial for many speech technology applications, including text-to-speech synthesis and synchronized facial animation. Manual segmentation is time-consuming, subjective, and prone to inconsistencies. While automatic methods like those based on HMMs are efficient, they often produce boundaries that are slightly off, leading to errors. This paper aims to bridge the gap between automatic and manual segmentation by refining HMM-based boundaries using acoustic-phonetic characteristics of speech sounds.

Methods

The proposed system employs a two-phase approach:

  1. Explicit Segmentation Module: This module uses Hidden Markov Models (HMMs) to align acoustic parameters of an utterance with phone models. The Viterbi algorithm is used to find the most likely sequence of phone states, which then defines initial phone boundaries. This is a standard approach in speech processing.
  2. Refinement Module: This is the core innovation. It takes the initial HMM-based segmentation and refines the boundaries. The refinement process leverages acoustic-phonetic features specific to different phone classes. The system groups phones into 11 classes (e.g., vowels, fricatives, plosives) and calculates specific parameters for each class. Thresholds for these parameters are determined from a manually segmented database. The refinement module then analyzes the acoustic-phonetic features within a defined "refinement interval" around each boundary and shifts the boundary to a position that better matches the acoustic characteristics of the transition between phone classes.
  3. Bias Removal Module (Optional): An additional module was added to address systematic biases in boundary placement that can occur with HMMs. This module estimates and removes average deviations between manual and automatic segmentations for specific class pairs.

Datasets

  • Speaker-Dependent (SDD) Brazilian Portuguese Speech Database: Used for training and evaluating the system.
  • Speaker-Independent (SID) TIMIT Database: A widely used benchmark dataset for speech research, also used for training and evaluation.

Key Insights and Ideas

  • Acoustic-Phonetic Features as Refiners: The paper demonstrates that specific acoustic-phonetic features, when analyzed in the context of phone classes, can effectively correct errors made by purely statistical models like HMMs.
  • Systematic Bias in HMMs: The study highlights that HMM-based segmentation can exhibit systematic biases, with boundaries sometimes appearing consistently too early or too late depending on the phone classes involved. The bias removal module addresses this.
  • Refinement Interval is Key: The refinement process works by analyzing a window of speech around a boundary. The size of this "refinement interval" is crucial for accurately identifying the true transition point.
  • SDD vs. SID Performance: The system performs better on the speaker-dependent database, which is expected given that the refinement thresholds were derived from it. However, it still shows significant improvement on the speaker-independent TIMIT database.

Key Results

The paper reports significant improvements in segmentation accuracy after applying the refinement module:

  • For the Speaker-Dependent (SDD) database:
    • Errors below 20 ms were reduced by approximately 30% (from 65.95% before refinement to 95.55% after refinement).
  • For the Speaker-Independent (SID) database:
    • Errors below 20 ms were reduced by approximately 13% (from 79.83% before refinement to 92.95% after refinement).
  • The bias removal module also provided improvements, though the refinement module alone achieved comparable results, suggesting its robustness.
  • The results are comparable to other state-of-the-art systems in the field.

Limitations and Future Work

  • Threshold Calculation: The refinement thresholds were primarily calculated using the speaker-dependent database. While they showed good performance on the speaker-independent database, further optimization might be possible.
  • Diphthong Handling: The paper suggests that new parameters might be needed to better handle diphthongs, especially those composed of two back or front vowels.
  • Context-Dependent Phones: The current system uses context-independent phone models. Testing with context-dependent phones is suggested for future work.
  • Sex-Independent Parameters: The authors propose exploring sex-independent parameters for broader applicability.
  • Bias Removal Necessity: While the bias removal module offers improvements, the authors note that it's not strictly necessary as the refinement process itself aims to find the optimal boundary. Further testing of different bias removal techniques is recommended.

Summary updated

BibTeX
@inproceedings{selmini2007acoustic,
  author = {Antonio Marcos Selmini and Fábio Violaro},
  title = {{Acoustic-phonetic features for refining the explicit speech segmentation}},
  booktitle = {Interspeech},
  year = {2007},
}

Loading related content...

Figure