TL;DR
This paper introduces a new frequency-domain method for analyzing and synthesizing voiced speech, improving robustness to recording imperfections like phase distortion. A listening test revealed that speech synthesized using this new method was generally preferred over traditional time-domain approaches, especially under challenging recording conditions.
What Problem Does This Paper Address?
Modeling speech production often relies on the ARX-LF model, which treats voiced speech as a vocal tract filter excited by a glottal source. Accurately estimating the parameters for this model, particularly the glottal source, has been a challenge. Traditional time-domain methods are highly sensitive to the precise timing of speech analysis frames and can be significantly degraded by phase distortion introduced by recording equipment. This sensitivity limits their applicability to speech recorded under ideal, phase-linear conditions. The authors aim to overcome these limitations by developing a more robust parameterization technique that is less affected by phase distortion and frame timing.
Methods
The core of this paper is a novel frequency-domain approach to ARX-LF parameterization. Building on prior work, this method analyzes the magnitude spectrum of speech signals, making it inherently robust to phase distortions.
Here's a breakdown of the key steps:
- Pitch-Synchronous Analysis: The analysis is performed on frames centered around pitch periods, ensuring consistency with the periodic nature of voiced speech.
- Spectral Estimation: Instead of relying solely on the Discrete Fourier Transform (DFT), the authors employ a least-squares harmonic analysis for more accurate spectral sample estimation.
- Vocal Tract Filter Estimation: An all-pole vocal tract filter is estimated for each frame using Discrete All-Pole Modeling (DAP), drawing from a codebook of possible LF model parameter configurations.
- Dynamic Programming for Continuity: To ensure smooth parameter transitions across an entire speech segment, a dynamic programming algorithm is used. This algorithm finds the sequence of LF model parameters and vocal tract filters that minimizes an overall error metric, considering factors like goodness-of-fit (Itakura-Saito error).
- LF Model Parameter Refinement: The LF pulse parameters are further refined using a magnitude spectrum fitting technique.
- Parameter Smoothing: Line spectral frequencies and R parameters are smoothed using an averaging filter to enhance temporal continuity.
For synthesis, the process is pitch-synchronous. Each synthesized speech pulse is generated by convolving an LF model pulse with the estimated vocal tract filter, windowing it, and then overlap-adding it to the output signal.
Datasets
The paper does not explicitly mention specific named datasets for training. However, the experiments for the preference test utilized:
- The first five sentences from the CMU-Arctic database for two speakers (one male, one female).
- Five sentences from a male and female speaker recorded using an inexpensive headset microphone and a laptop computer.
The test signals were manipulated to simulate different recording conditions:
- Linear Phase: Signals recorded using phase-linear equipment.
- Non-linear Phase: Signals convolved with the impulse response of a non-linear phase recording device (described in reference [18]).
- Laptop: Signals recorded with a generic non-linear phase audio recording setup.
Key Insights and Ideas
- Frequency Domain Robustness: The most significant insight is the demonstrated robustness of the frequency-domain approach to phase distortion. By focusing on spectral magnitudes, the method bypasses the phase-related issues that plague time-domain techniques, making it more practical for real-world recordings.
- Preference for Naturalness: The listening test results strongly suggest that listeners generally prefer speech synthesized using the frequency-domain ARX-LF parameters, indicating a more natural or pleasing output, especially under non-ideal recording conditions.
- Challenges with Male Voices in Linear Conditions: Interestingly, the preference for the frequency-domain method was less pronounced for male voices recorded under ideal linear phase conditions. This might suggest that time-domain methods perform relatively well in these perfect scenarios, or that the frequency-domain method introduces subtle differences that are less desirable when the original signal is pristine.
- Female Voice Advantage: The frequency-domain approach showed a particular advantage for female voices, even under linear phase conditions. The authors hypothesize this could be due to the inherent difficulty in accurately estimating glottal closing instants for female voices in time-domain methods.
Key Results
The primary result is from the listening test involving 50 participants. The key findings were:
- General Preference: The frequency-domain ARX-LF parameterization approach was generally preferred over the time-domain method across all recording conditions and for both male and female speakers.
- Stronger Preference in Non-Ideal Conditions: The preference for the frequency-domain method was significantly higher under non-linear phase conditions (simulated and real-world inexpensive equipment) compared to linear phase conditions.
- Slight Preference for Male Voices in Linear Phase: For male speakers under linear phase conditions, the preference was not as clear, indicating a more comparable performance between the two methods in this specific scenario.
- Qualitative Observation: Speech synthesized by the frequency-domain method was perceived as "brighter" and containing more high-frequency energy compared to the time-domain method. The authors suggest this might be because time-domain methods, driven by least-squares error, tend to prioritize fitting lower-frequency formants at the expense of higher-frequency details.
Limitations and Future Work
- Limitations:
- The paper acknowledges that the ARX-LF model itself, when applied to speech synthesis alone, may produce synthesized speech that differs from the original signal, even if the parameters are well-estimated.
- The authors note that the time-domain ARX-LF parameterization method used for comparison had some errors leading to "disagreeable discontinuity-type artifacts" in a few sentences, which were removed from the dataset.
- Future Work:
- The authors state that future work will focus on developing a robust system for ARX-LF model-based voiced speech modification. This implies extending the current parameterization to enable more sophisticated manipulation of speech characteristics.
Summary updated