Acoustic segmentation of speech using zero time liftering (ZTL)

RaviShankar Prasad, B. YegnanarayanaInterspeech 2013

TL;DR

This paper introduces a novel, unsupervised method for segmenting speech signals into meaningful acoustic units. By leveraging a technique called Zero Time Liftering (ZTL) and analyzing Dominant Resonant Frequencies (DRFs), the authors achieve segmentation comparable to existing supervised methods without requiring labeled data. This advancement is crucial for various speech processing applications that rely on accurate acoustic boundary detection.

What Problem Does This Paper Address?

Accurately dividing speech into its constituent acoustic segments (like phonemes or syllables) is a fundamental challenge in speech processing. This "segmentation" is vital for applications such as speech recognition, phonetic analysis, and speech coding. Traditionally, the most precise segmentation relies on manual annotation of speech data, which is time-consuming and expensive. While Hidden Markov Models (HMMs) have been widely used for automatic segmentation, they typically require large amounts of transcribed data (supervised learning). Unsupervised methods, which don't need labeled data, have historically struggled to match the accuracy of supervised approaches, often facing challenges with consistency and temporal resolution. This paper aims to bridge this gap by proposing an unsupervised method that achieves high accuracy and temporal resolution.

Methods

The core of the proposed method lies in two key techniques:

  • Zero Time Liftering (ZTL): This is a signal processing technique that provides high temporal resolution while maintaining good spectral resolution. It achieves this by multiplying the speech signal with a specific decaying window function and then analyzing the Hilbert envelope of the Numerator Group Delay (HNGD) of the signal. This process effectively highlights high Signal-to-Noise Ratio (SNR) regions in the spectrum, which are less affected by external factors and are thus more robust for analysis.
  • Dominant Resonant Frequencies (DRFs): By applying ZTL analysis to speech segments, the authors extract the frequencies of the dominant peaks in the HNGD spectra. These DRFs are interpreted as representing the production characteristics of the speech signal, essentially reflecting the shape of the vocal tract at a given moment. Changes in these DRFs are used to identify boundaries between different acoustic events.

The segmentation algorithm works by first differentiating between "obstruent" and "sonorant" regions based on DRF characteristics. For sonorant regions, a 3-point median filter is applied to smooth the DRF curves. The algorithm then identifies transitions in DRF locations, controlled by a parameter ftr, to mark segment boundaries. Multiple boundaries within a short time window (20ms) are merged to avoid ambiguity.

Datasets

The primary dataset used for evaluation is the TIMIT database. The authors specifically mention using a subset of 182 sentences from TIMIT, which is a widely used corpus for phone segmentation tasks. They note that TIMIT consists of recordings from 630 American-English speakers at a sampling rate of 16kHz. No new benchmarks were introduced; the evaluation was performed on an existing, well-established dataset.

Key Insights and Ideas

  • DRFs as Vocal Tract Fingerprints: The paper highlights that Dominant Resonant Frequencies (DRFs) derived from ZTL analysis provide a powerful, data-driven way to represent the instantaneous production characteristics of speech, directly linked to vocal tract shape.
  • Unsupervised Segmentation with High Resolution: The combination of ZTL and DRF analysis offers a path to unsupervised speech segmentation that achieves temporal resolution comparable to manual segmentation, a significant improvement over many existing unsupervised methods.
  • Robustness to Language and Speaker: The DRF-based approach is presented as being independent of language, speaker, and corpus-specific dependencies, making it broadly applicable.
  • Potential for Improved Accuracy: The authors suggest that manual transcriptions might not always be perfectly accurate and that their DRF-based method, by focusing on acoustic event transitions, could potentially be more accurate in some instances.

Key Results

The paper reports segmentation performance using the Correct Detection Rate (CDR), denoted by α, which measures the percentage of correctly detected boundaries within a given tolerance.

  • Performance on TIMIT: For a tolerance level of 20ms, the proposed DRF-based unsupervised method achieved a CDR of 79.6%.
  • Comparison with Supervised Methods: At a 20ms tolerance, the DRF method's CDR (79.6%) is competitive with some supervised methods, though it falls short of the top-performing supervised methods like HMM+SVM (95.4%).
  • Comparison with Other Unsupervised Methods: The DRF method outperforms other unsupervised methods like MMC (67.9%), AP seg (77.2%), and DSF (77.2%) at a 20ms tolerance.
  • Over-segmentation Rate (OSR): The algorithm exhibits an over-segmentation rate (β) of around 11% for non-silence regions.

Limitations and Future Work

  • Gap with Supervised Methods: While the DRF method performs well for an unsupervised approach, there is still a gap in performance compared to the best supervised methods. The authors acknowledge that further refinements are needed to bridge this gap.
  • Future Work: The authors plan to:
  • Incorporate other speech production-based features alongside DRFs to further improve segmentation performance.
  • Develop methods to automatically label the identified acoustic segments into different phonetic or linguistic categories.

Summary updated

BibTeX
@inproceedings{prasad2013acoustic,
  author = {RaviShankar Prasad and B. Yegnanarayana},
  title = {{Acoustic segmentation of speech using zero time liftering (ZTL)}},
  booktitle = {Interspeech},
  year = {2013},
}
Figure