Abstract
O artigo compara modelos de síntese de fala com arquiteturas baseadas em espectrograma e fim-a-fim, com o objetivo de determinar a capacidade de clonagem de voz em cenário low-resource. Foram avaliados conjuntos de treinamento de adaptação com diferentes quantidades de fala para clonagem de uma voz alvo, e o tempo necessário para realizar o treinamento. O modelo VITS mostrou-se mais eficiente, alcançando os melhores resultados no teste de qualidade perceptual no cenário low-resource com dados no idioma português, e completou o treinamento em menos tempo, quando comparado com o Tacotron2.
Introduction
Creating high-quality Text-to-Speech (TTS) systems traditionally requires extensive datasets, often exceeding dozens of hours of professional recordings. However, in many practical applications—such as personalized digital assistants or voice restoration—only limited data (low-resource) is available. This study investigates the efficiency of two prominent generative architectures, Tacotron2 and VITS, in cloning a target voice for Brazilian Portuguese using as little as five minutes of audio.
Methodology
The research evaluates two distinct paradigms in speech synthesis:
- Spectrogram-based (Tacotron2): A two-stage pipeline where text is first converted into a mel-spectrogram, which is then transformed into a waveform by a separate vocoder (HiFi-GAN).
- End-to-End (VITS): A single-stage architecture using Variational Inference to map text directly to audio waveforms, bypassing intermediate spectrogram generation.
Training Strategy
The models underwent a two-phase training process:
- Base Training: Models were pre-trained on the TTS-Portuguese Corpus (10.5 hours of a single male speaker) to learn linguistic fundamentals.
- Fine-tuning (Cloning): The base models were adapted to a new target male speaker using subsets of a 20-minute proprietary dataset (CPQD), specifically testing 20, 15, 10, and 5-minute segments.
Results and Discussion
The evaluation combined objective metrics like Mel-Cepstral Distortion (MCD) with subjective human assessments. VITS consistently outperformed Tacotron2 across all data sizes.
As shown in the figure above, the subjective ratings (on a 0-100 scale) indicate that VITS (represented by the C2 stimulus) achieved higher median scores and lower variance compared to Tacotron2 (C1). While Tacotron2 often exhibited "glitches" or artifacts when trained on minimal data, VITS maintained a robust output quality. Specifically, with only 5 minutes of data, VITS achieved a score of 71.49, significantly higher than Tacotron2's 67.49.
Furthermore, VITS demonstrated superior efficiency in training time. High-quality cloning was achievable after just one hour of fine-tuning, peaking at 20 hours. This efficiency, combined with the lack of error accumulation inherent in two-stage systems, makes VITS a more practical solution for low-resource scenarios.
Conclusion
The study concludes that end-to-end architectures like VITS are significantly more effective for low-resource voice cloning in Brazilian Portuguese than traditional spectrogram-based models. VITS provides higher naturalness, better voice similarity, and faster convergence. Future research will explore cross-gender cloning and the impact of multi-speaker base models to further reduce the data requirements for high-fidelity voice synthesis.
Summary updated