Korean Singing Voice Synthesis Based on Auto-Regressive Boundary Equilibrium Gan

Soonbeom Choi, Wonil Kim, Saebyul Park, Sangeon Yong, Juhan NamICASSP 2020 · OpenReview

Abstract

Singing voice synthesis is a generative task that involves not only multidimensional controls of a singer model such as phonetic modulation by lyrics and pitch control by music score but also expressive elements such as breath sounds and vibrato. Recently, end-to-end learning models based on generative adversarial network (GAN) have drawn much interest as it requires less domain-specific processing but provides high sound quality. When GAN is applied to the audio domain, it entails several issues: the choice of audio representation to generate, handling temporal continuity between two adjacent outputs, and finding an effective loss metric for the audio representation. In this paper, we propose a Korean singing voice synthesis system that addresses the issues using an auto-regressive algorithm that generates spectrogram with the boundary equilibrium GAN objective. Through the qualitative test, we show the proposed methods are superior to the original GAN objective and non-auto-regressive model. We also show that our proposed method can render natural expressions such as continuous pitch contours and breath sounds.
BibTeX
@inproceedings{choi2020korean,
  author = {Soonbeom Choi and Wonil Kim and Saebyul Park and Sangeon Yong and Juhan Nam},
  title = {{Korean Singing Voice Synthesis Based on Auto-Regressive Boundary Equilibrium Gan}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=24TkoaLYU1t},
}

Loading related content...

Figure