Quasi-Periodic WaveNet Vocoder: A Pitch Dependent Dilated Convolution Model for Parametric Speech Generation

Yi-Chiao Wu, Tomoki Hayashi, Patrick Lumban Tobing, Kazuhiro Kobayashi, Tomoki TodaInterspeech 2019

TL;DR

This paper introduces the Quasi-Periodic WaveNet (QPNet) vocoder, which enhances pitch controllability in neural vocoders by incorporating a novel pitch-dependent dilated convolution (PDCNN) mechanism. This allows QPNet to generate speech with more accurate and controllable pitch, even for frequencies outside the training range, while maintaining comparable speech quality to existing models.

What Problem Does This Paper Address?

Traditional parametric speech synthesis systems, known as vocoders, offer high flexibility in manipulating synthesized speech, including pitch. However, modern neural vocoders like WaveNet (WN), while achieving impressive speech naturalness, often struggle with pitch controllability. This limitation stems from their fixed network architectures, which make it difficult to generate speech with fundamental frequencies (Fo) that fall outside the range observed during training. This deficiency hinders their application in scenarios requiring precise pitch manipulation, a capability essential for many speech synthesis tasks.

Methods

The core innovation of this paper is the Pitch-Dependent Dilated Convolution (PDCNN) module, integrated into a Quasi-Periodic WaveNet (QPNet) vocoder.

  • WaveNet (WN) Baseline: The paper starts by describing the standard WaveNet vocoder, an autoregressive model that predicts speech samples based on previous samples and auxiliary acoustic features. It utilizes stacked dilated causal convolutions to efficiently extend its receptive field, enabling it to model long-term dependencies in speech.
  • Pitch-Dependent Dilated Convolution (PDCNN): This novel component adapts the dilation size of the convolutional filters based on the input fundamental frequency (Fo). Instead of a fixed dilation size, PDCNN dynamically adjusts it to ensure that each speech sample's receptive field corresponds to a specific number of previous cycles, rather than a fixed number of samples. This is achieved by introducing a pitch-dependent dilated factor, $\epsilon_t = F_s / (F_{o,t} \times \alpha)$, where $F_s$ is the sampling rate, $F_{o,t}$ is the fundamental frequency at time $t$, and $\alpha$ is a hyperparameter related to the pitch cycle length.
  • Cascaded Architecture: QPNet employs a two-module cascaded structure. The first module is a standard WaveNet that captures short-term correlations. The second module incorporates the PDCNN to model long-term periodic correlations, inspired by the source-filter model and CELP codecs. This separation allows for more effective modeling of both periodic and non-periodic speech components.

Datasets

  • Training Data: The models were trained on a combination of the CMU-ARTIC dataset ('bdl' and 'slt' speakers) and all training data from the VCC2018 dataset.
  • Evaluation Data: The evaluation set consisted of 35 testing utterances per speaker from the SPOKE set of VCC2018 (four source speakers: two male, two female).
  • Fo Transformation: To evaluate performance on unseen Fo values, the original Fo was scaled by ten different ratios ranging from 1/2 to 2.

Key Insights and Ideas

  • Pitch Controllability is Key: The paper strongly emphasizes that while WaveNet excels at naturalness, its lack of pitch controllability is a significant drawback for many applications.
  • Fo-Dependent Receptive Fields: The core idea is that the temporal dependencies in speech are not uniform; they are strongly influenced by the pitch period. By making the receptive field size dependent on the Fo, the model can more effectively capture these pitch-related structures.
  • Efficiency Gains: The pitch-dependent dilation allows for a more efficient expansion of the receptive field, potentially leading to smaller network sizes for comparable performance, as demonstrated by the QPNet's performance relative to a larger WN.
  • Bridging Traditional and Neural Vocoders: QPNet aims to combine the flexibility of traditional vocoders (pitch control) with the naturalness of neural vocoders (WaveNet).

Key Results

  • Pitch Accuracy: QPNet significantly outperforms a similarly sized WaveNet (WNc) in terms of root mean square error (RMSE) of logarithmic Fo. It also achieves better pitch generation accuracy than a larger WaveNet (WNf), especially when conditioning on scaled Fo values.
  • Spectral Distortion: While QPNet shows improved pitch accuracy, it exhibits slightly worse mel-cepstral distortion (MCD) compared to the larger WNf, suggesting a potential trade-off between pitch control and spectral fidelity at very short receptive field lengths. However, it still significantly improves spectral prediction compared to the original WNc.
  • Subjective Evaluation (MOS): In the unchanged Fo range, QPNet achieved comparable sound quality to WNf, surpassing WNc. In outside Fo ranges (1/2 Fo), QPNet maintained acceptable quality and showed remarkable pitch accuracy.
  • Efficiency: QPNet requires approximately 75% of the training time and 40% of the generation time compared to a full-sized WaveNet, due to its more compact network size.

Limitations and Future Work

  • Receptive Field Length Trade-off: The paper acknowledges that the compact network size of QPNet, while efficient, leads to shorter receptive field lengths, which can degrade spectral prediction ability, especially for higher Fo scaling ratios.
  • Future Work: The authors plan to investigate the effectiveness of QPNet in voice conversion tasks.

Summary updated

BibTeX
@inproceedings{wu2019quasi,
  author = {Yi-Chiao Wu and Tomoki Hayashi and Patrick Lumban Tobing and Kazuhiro Kobayashi and Tomoki Toda},
  title = {{Quasi-Periodic WaveNet Vocoder: A Pitch Dependent Dilated Convolution Model for Parametric Speech Generation}},
  booktitle = {Interspeech},
  year = {2019},
}

Loading related content...

Figure