TL;DR
This paper investigates the perceptual importance of phase information in synthetic speech, a factor often neglected in favor of spectral magnitude. By using a novel Relative Phase Shift (RPS) representation, the authors demonstrate that even subtle phase manipulations can lead to audible degradations in speech quality, suggesting that more sophisticated phase modeling is crucial for high-fidelity speech synthesis.
What Problem Does This Paper Address?
Speech synthesis systems often simplify or discard the original phase information of speech signals, assuming it has minimal impact on perceived quality. This is partly due to the complexity of modeling and manipulating phase. While spectral magnitude (which relates to the signal's energy at different frequencies) is well-understood and widely used, phase information, which governs the temporal structure of the signal, has historically been treated as less critical. Early theories by Ohm and Helmholtz suggested that only spectral power mattered for sound quality. Although later research has shown that phase can be perceptible, especially in certain frequency ranges and for specific signal types, its precise role in the perceptual quality of synthetic speech remains a subject of debate. This paper aims to definitively answer whether phase information deserves more elaborate modeling in speech synthesis to achieve the highest possible quality, or if its subtle effects can be safely overlooked.
Methods
The core of this work relies on the Relative Phase Shift (RPS) representation, a novel method proposed by the authors for analyzing and manipulating phase information in speech. RPS represents the phase difference between each harmonic component and the fundamental frequency at a specific point in the signal's period. This representation offers a clear view of the phase structure and allows for straightforward manipulation.
The authors then explore several phase manipulation techniques, some inspired by common practices in speech vocoders:
- Polarity Inverted Signals: A basic test to see if simple polarity flips are perceptible in speech.
- Time-Constant RPSS: Replacing original RPS with constant values. This includes:
- Zero Phase (Cosine-Phase): Setting all RPS values to zero.
- Random Constant RPS: Assigning a random constant value to each harmonic.
- Minimum Phase: Converting the original spectrum to minimum phase before RPS analysis. This is a common simplification in LPC-based vocoders.
- Phase Models: Two models were evaluated for their ability to represent phase:
- Linear Model: A very basic model where the unwrapped RPS is treated as a linear function of frequency.
- DCT-mel-RPS Model: A more sophisticated model, previously used in ASR, that involves mel-filtering, DCT transformation, and inverse transformations.
These manipulated phases are then used to resynthesize speech signals using a harmonic plus stochastic coding system (HSM-iMBA), with the stochastic component removed to isolate phase effects. The evaluation uses a double blind triple stimulus with hidden reference method, where listeners compare manipulated signals against an original, resynthesized reference.
Datasets
The evaluation corpus consists of voiced-only speech signals recorded by 6 non-professional Spanish speakers (3 male, 3 female). Each speaker recorded 10 sentences, and 8 of these were selected for the evaluation. This focus on voiced segments is intended to minimize the influence of stochastic components and allow listeners to concentrate on phase-related timbre variations.
Key Insights and Ideas
- Phase Matters, Even Subtly: Contrary to the assumption that phase is largely inaudible in speech, the study reveals that even minor phase manipulations can lead to detectable degradations in perceived speech quality.
- RPS as a Powerful Tool: The RPS representation proves to be a valuable tool for analyzing phase structure and enabling controlled manipulation, facilitating this investigation.
- Male Voices More Sensitive: Male voices appear to be more affected by phase manipulation than female voices, potentially due to a higher concentration of low-frequency components where phase perception is known to be more acute.
- Common Vocoder Simplifications Cause Degradation: Standard techniques like assuming zero or random phase, and even minimum phase, introduce noticeable impairments.
- Sophisticated Models Perform Better: The DCT-mel-RPS model, a more advanced phase representation, yields significantly better results, producing only slight, often imperceptible, impairments.
Key Results
The evaluation design was successful in making phase effects perceptible.
- Detection Rate: 75.8% of manipulated signals were correctly detected as manipulated.
- Degradation Scores: All tested phase manipulations resulted in worse scores than the original, unmodified signal.
- Rough Manipulations: Zero-phase, random constant phase, and linear models showed no significant differences among themselves but all indicated substantial degradation.
- Minimum Phase: Produced a remarkable but mostly not annoying degradation.
- DCT-mel-RPS Model: Performed very well, resulting in slight and not annoying impairments.
- Polarity Inversion: Surprisingly, polarity inversion was sometimes detectable and caused a very slight, statistically significant impairment.
Limitations and Future Work
The authors acknowledge that the evaluation design was specifically crafted to maximize the perception of phase effects. Therefore, the observed degradations might be less pronounced in everyday speech signals. They also note that the resynthesis process itself can introduce artifacts, and that the harmonic assumption might mismodel stochastic energy, leading to perceived tonal noises.
Future work could involve exploring more advanced phase modeling techniques and investigating the perceptual impact of phase in a wider range of speech conditions, including unvoiced segments and different speaker populations. The findings strongly suggest that incorporating more elaborate phase modeling into speech synthesis systems is a promising avenue for achieving higher perceptual quality.
Summary updated