TL;DR
This paper introduces a novel representation of speech phase information called the Relative Phase Shift (RPS) and demonstrates its ability to improve Automatic Speech Recognition (ASR) performance. By incorporating RPS features alongside traditional Mel-Frequency Cepstral Coefficients (MFCCs), the researchers achieved significant reductions in word error rates, suggesting that phase information, often overlooked, holds valuable cues for speech recognition.
What Problem Does This Paper Address?
Automatic Speech Recognition (ASR) systems typically rely on spectral magnitude information (like MFCCs) to model speech. While spectral phase contains information about the timing and structure of speech, it's notoriously difficult to model due to its "wrapping" nature and the perceived controversy surrounding its contribution to intelligibility. Previous attempts to incorporate phase information have yielded inconsistent results. This paper aims to address this by proposing and evaluating a new phase representation, the Relative Phase Shift (RPS), which is designed to be more amenable to modeling and to capture structured patterns in speech phase.
Methods
The core of this paper is the introduction and parameterization of the Relative Phase Shift (RPS). The RPS is derived from the harmonic analysis of speech, focusing on the differences between the initial phase shifts of harmonically related sinusoids that model a speech frame. This relative approach makes the representation robust to local stationarity assumptions.
Key methodological steps include:
- RPS Derivation: Calculating the difference between instantaneous phases of harmonically related sinusoids, normalized by their harmonic number.
- Unwrapping: Addressing the [-π, π] wrapping of RPS values to obtain a smoother, continuous function.
- Differentiation: Applying differentiation to the unwrapped RPS to highlight phase jumps, which are often related to vocal tract formants.
- Mean Normalization: Removing speaker and channel-dependent offsets from the differentiated RPS.
- Parameterization: Employing the Discrete Cosine Transform (DCT) to reduce dimensionality and create compact feature sets. Two main parameterization strategies were explored:
- Frequency Interpolation: Resampling unwrapped RPS "envelopes" at fixed frequencies.
- Mel Filter Bank: Applying a Mel filter bank to the RPS data, inspired by its success in spectral energy-based ASR.
- Feature Fusion: Combining the derived RPS features with standard MFCCs for ASR experiments.
- Polarity Correction: Using a dedicated algorithm to ensure consistent signal polarity before RPS extraction, as RPS is sensitive to it.
Datasets
- Basque Language Speech Database: A multi-speaker database (230 speakers, 127 female, 103 male) recorded in home and office environments, sampled at 16 kHz and 16 bits.
- Training: 43,953 utterances.
- Testing: 11,859 utterances.
- Dictionary: 435 words.
Key Insights and Ideas
- Phase Holds Valuable Information: The study strongly suggests that spectral phase, when appropriately represented and parameterized, can significantly improve ASR performance, challenging the notion that it's too difficult to model or contributes little to intelligibility.
- RPS Captures Structured Patterns: The RPS phasegram reveals clear, stable patterns in voiced speech segments, which are visually distinct for different vowels and speakers, hinting at its potential for speech analysis.
- Differentiated RPS Highlights Formants: Differentiating the unwrapped RPS makes phase jumps more prominent, aligning them with vocal tract formant frequencies, a key characteristic of speech.
- Mel-Filtering Enhances RPS: Similar to its effectiveness with spectral energy, applying a Mel filter bank to RPS features proved to be a highly effective parameterization strategy for ASR.
- Dynamic MFCCs Overlap with Phase Information: When dynamic features (deltas and accelerations) are added to MFCCs, the improvement from RPS becomes less pronounced, suggesting that dynamic MFCCs capture some of the temporal information already present in the phase features.
Key Results
The paper reports significant improvements in Word Error Rate (WER) when RPS features are combined with MFCCs:
- Best Performance: The combination of MFCCs with Mel-filtered, mean-normalized, frequency-differenced RPS achieved a 22.69% reduction in WER compared to the MFCC-only baseline.
- With Dynamic Features: Even when using dynamic MFCC features (deltas and accelerations), the best RPS parameterization still yielded a 12.31% reduction in WER.
- RPS Alone: When RPS features (with MFCC0 for segmentation) were used alone, they achieved a recognition rate of 67.1%, which is considered good given that RPS carries no information for unvoiced sounds.
- Statistical Significance: The best results were found to be statistically significant (p<0.05).
Limitations and Future Work
- Unvoiced Speech: The paper acknowledges that RPS is not meaningful for unvoiced speech segments, as there's no clear fundamental frequency (F0) reference. While a workaround is proposed (arbitrary F0), it doesn't imbue these segments with meaningful phase information.
- Direct Comparison: The authors note that direct comparison with other published phase-based ASR results is difficult due to differing experimental setups and datasets.
- Future Work: The authors plan to:
- Extensively evaluate RPS usefulness with different databases, acoustic conditions, and languages.
- Apply RPS to more complex ASR tasks.
- Further explore the potential of RPS in other speech processing applications.
Summary updated