A comparison of broad phonetic and acoustic units for noise robust segment-based phonetic recognition

Tara N. Sainath, Victor ZueInterspeech 2008

TL;DR

This paper introduces a novel approach to improve noise robustness in speech recognition by using broad phonetic and acoustic units as a pre-processing step for landmark detection. By comparing these broad units, the authors demonstrate significant improvements in phonetic recognition accuracy on noisy data and identify trade-offs in computation time depending on noise type.

What Problem Does This Paper Address?

Modern speech recognition systems rely on breaking down speech into smaller units, typically phonemes, to map acoustic signals to words. While phonemes are linguistically well-defined, their acoustic realization can vary significantly, especially in noisy environments. Existing methods for segmenting speech and detecting important acoustic "landmarks" (points of significant change) struggle when noise corrupts the signal, leading to poor recognition performance. This paper aims to improve noise robustness by exploring whether using broader, acoustically or phonetically motivated units can lead to more stable landmark detection and better overall speech recognition in the presence of noise.

Methods

The core of this work lies in using "broad classes" of speech sounds as a pre-processing step for landmark detection within the SUMMIT segment-based speech recognition framework.

  • Broad Phonetic Units (BPCs): These are groups of phonemes that are acoustically similar and tend to be invariant in noise. Examples include vowels, stops, fricatives, etc.
  • Broad Acoustic Units (BACs): These are learned directly from the acoustic signal by segmenting utterances into quasi-stationary sections and then clustering these segments. This process uses an agglomerative clustering approach, starting with pre-clustered "seed" clusters.
  • Novel Cluster Evaluation (V-Measure): To determine the optimal number of BACs, the authors adapt the V-measure, a metric that balances homogeneity (clusters containing only one class) and completeness (all members of a class are in the same cluster). They introduce a phonetic similarity component to this V-measure to better account for acoustically similar but distinct phonetic classes.
  • Landmark Detection and Segmentation: Transitions between these broad classes (BPCs or BACs) are used as anchor points for hypothesizing major landmarks. Minor landmarks are then placed based on spectral change. This broad-class-guided landmark detection is then used to build a segment graph for recognition.

Datasets

  • TIMIT Corpus: Used for training and evaluating the phonetic recognition models. The corpus provides phonetically rich context and hand-labeled transcriptions.
  • Noisex-92 Database: Used to simulate noisy conditions by adding pink, speech babble, and factory noise to the TIMIT corpus at various Signal-to-Noise Ratios (SNRs) ranging from 30dB to -5dB.

Key Insights and Ideas

  • Broad classes offer robustness: Grouping phonemes into broader phonetic or acoustic categories provides a more stable representation in noise compared to individual phonemes.
  • Acoustic vs. Phonetic: Both BPC and BAC approaches significantly outperform baseline spectral change and sinusoidal methods in noisy conditions.
  • Noise type matters for speed: The choice between BPC and BAC has implications for computation time. BACs are faster in stationary noises, while BPCs are faster in non-stationary noises. This suggests that the nature of the noise influences which broad unit representation is more computationally efficient.
  • Phonetic similarity is key for BACs: Incorporating phonetic similarity into the V-measure for evaluating BACs helps select a more appropriate number of clusters, especially at lower SNRs where phonetic confusions increase. This leads to better-quality acoustic clusters.

Key Results

  • Improved Accuracy: The BPC and BAC segmentation methods achieved significantly lower Phonetic Error Rates (PERs) compared to the baseline spectral change and sinusoidal methods across various noise types and SNRs on the TIMIT dataset. For example, at -5dB SNR, the BAC method achieved a PER of 69.6%, while the spectral change method reached 91.8%.
  • Landmark Robustness: The BPC and BAC methods produced more robust landmarks, with smaller time deviations from true phoneme boundaries compared to the baselines, especially at lower SNRs.
  • Computation Time Trade-offs:
    • BAC methods were faster in stationary noises (e.g., pink noise).
  • BPC methods were faster in non-stationary noises (e.g., babble, factory noise).
  • V-Measure Effectiveness: The modified V-measure with phonetic similarity helped select a more appropriate number of BAC clusters, particularly at lower SNRs, indicating better acoustic distinctiveness.

Limitations and Future Work

  • Limitations: The paper notes that at very low SNRs (-5dB babble noise), where harmonics are poorly preserved, the BAC performance degrades. The authors also mention that while BPC and BAC performance is similar, their computation times differ.
  • Future Work: The authors suggest that preliminary results on the Aurora-2 noisy digit task using BAC and BPC approaches show promise and indicate the generalizability of these methods to other tasks. Further investigation into the specific acoustic properties that make BACs faster in stationary noise and BPCs faster in non-stationary noise would be beneficial.

Summary updated

BibTeX
@inproceedings{sainath2008comparison,
  author = {Tara N. Sainath and Victor Zue},
  title = {{A comparison of broad phonetic and acoustic units for noise robust segment-based phonetic recognition}},
  booktitle = {Interspeech},
  year = {2008},
}
Figure