Speech recognition using long-term phase information

Kazumasa Yamamoto, Eiichi Sueyoshi, Seiichi NakagawaInterspeech 2010

TL;DR

This paper proposes using long-term phase information, specifically group delay features, for speech recognition, challenging the conventional reliance on amplitude spectrum features. Their experiments demonstrate that phase information alone can achieve high word and digit recognition accuracy, and combining it with amplitude features offers significant improvements.

What Problem Does This Paper Address?

Traditional speech recognition systems primarily utilize amplitude spectrum features, such as Mel-Frequency Cepstral Coefficients (MFCCs), largely discarding the phase spectral information. While human speech perception studies suggest phase plays a crucial role, its direct application in automatic speech recognition (ASR) has been limited. This paper investigates whether long-term phase information, analyzed over extended time windows (greater than 100 ms), can provide valuable linguistic cues for ASR, aiming to improve recognition performance by incorporating this underutilized feature.

Methods

The core of this paper lies in extracting and utilizing "long-term phase spectral features." The authors propose two main approaches:

  • Delta Phase Parameters for Frequency as Group Delay Features: This method approximates the derivative of the unwrapped phase spectrum with respect to frequency. They use a specific formula (Equation 4) to calculate this "group delay" over a window, aiming to capture how the phase changes across frequencies.
  • Analytic Group Delay Features: This approach uses a more formal definition of group delay derived from the complex spectrum and its time-multiplied counterpart (Equation 5). For recognition, they further process these features by applying a moving average and then a Discrete Cosine Transform (DCT) to create "Long-Term Group Delay Cepstrum" (LTGDC) features.

These phase-based features are then used either independently or in combination with standard MFCCs for speech recognition experiments.

Datasets

The paper uses two primary datasets for evaluation:

  • CIAIR-VCV: An in-room child speech database used for isolated word recognition experiments. It contains approximately 30 word utterances per speaker, with 125 speakers for training and 20 for testing.
  • CENSREC-1 (AURORA-2J): A Japanese version of the AURORA-2 database, used for continuous digit recognition experiments. This dataset includes clean speech and speech corrupted by various noise types at different Signal-to-Noise Ratios (SNRs). The clean training dataset consists of 8440 utterances from 110 speakers.

Key Insights and Ideas

  • Phase Isn't Just Noise: The most significant insight is that long-term phase information, when analyzed appropriately, contains substantial linguistic information sufficient for speech recognition, challenging the long-held assumption that only amplitude matters.
  • Long-Term Analysis is Key: The effectiveness of phase features is strongly dependent on the analysis window length. Short windows (comparable to MFCC analysis) yield poor results, while longer windows (e.g., 100-256 ms) lead to high accuracy, suggesting that phase information accumulates and becomes more discriminative over longer durations.
  • Group Delay as a Powerful Feature: The proposed group delay-based features (both delta phase and analytic group delay cepstrum) demonstrate competitive performance, especially when used with long analysis windows.
  • Synergy Between Amplitude and Phase: Combining MFCCs (amplitude-based) with LTGDC (phase-based) features leads to significant performance improvements, particularly for cleaner speech, highlighting the complementary nature of these feature types.

Key Results

  • Isolated Word Recognition: Using only delta phase parameters (Διƒ) with a 256 ms window achieved over 96% accuracy, and LTGDC features also reached similar high accuracies.
  • Continuous Digit Recognition: For clean speech, LTGDC features alone achieved over 90% digit accuracy.
  • Combined Features: Combining MFCCs and LTGDC features resulted in a relative improvement of over 10% for higher SNR conditions, and even over 20% for clean speech in the CENSREC-1 dataset.
  • Robustness to Noise: LTGDC features showed better performance than MFCCs in certain noisy conditions (e.g., "Babble," "Restaurant," "Airport"), which are characterized by human voices, suggesting phase might be less affected by these types of noise.

Limitations and Future Work

  • Lombard Effect: The paper acknowledges that the Lombard effect (speech changes due to noise) was not considered in the CENSREC-1 experiments, as the data was simulated. Future work should investigate performance in real-world noisy environments to see how the Lombard effect impacts phase information.
  • Feature Domain Combination: The authors suggest exploring combining MFCC and LTGDC features in the feature domain, rather than just likelihoods, for potential further gains.
  • Large Vocabulary Continuous Speech Recognition (LVCSR): Applying LTGDC features to LVCSR tasks is identified as a promising future direction.
  • Modified Group Delay: The paper briefly mentions a "modified group delay" (Equation 7) proposed by Kua et al. and expresses an intention to consider it in future work.

Summary updated

BibTeX
@inproceedings{yamamoto2010speech,
  author = {Kazumasa Yamamoto and Eiichi Sueyoshi and Seiichi Nakagawa},
  title = {{Speech recognition using long-term phase information}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure