Introduction
Speaker adaptation is a critical task in Text-to-Speech (TTS) synthesis, enabling models to mimic a target speaker's voice using limited reference audio. Existing techniques primarily rely on zero-shot encoding or model fine-tuning. Zero-shot methods, while fast, often fail to capture the nuanced timbre of the target speaker. Conversely, fine-tuning on sparse data (e.g., 1–5 sentences) frequently results in catastrophic overfitting, where the model gains speaker similarity at the expense of speech quality and intelligibility.
To address these limitations, we propose ASRRL-TTS, an Agile Speaker Representation Reinforcement Learning framework. Instead of modifying the underlying TTS model weights, ASRRL-TTS employs a Reinforcement Learning (RL) agent to search for an optimal speaker embedding that maximizes similarity, quality, and intelligibility.
Methodology
Framework Overview
The ASRRL-TTS framework decouples the speaker's timbre from the linguistic content by refining the speaker representation in a frozen TTS model. The process involves state initialization, an RL-based action design, and a multi-dimensional reward mechanism.
Reinforcement Learning for Embedding Optimization
The core of the system is an RL agent trained via Proximal Policy Optimization (PPO). The agent's state $s$ is defined by the concatenation of textual features $f_t$ (extracted via a pre-trained BERT model) and the initial speaker embedding $e$. This inclusion of textual context allows the agent to make context-aware adjustments to the speaker representation.
The agent performs actions based on the available data:
- Single-Sentence (SS) Scenario: The agent generates a refinement vector added to the initial embedding.
- Few-Sentence (FS) Scenario: When multiple reference clips are available, the agent learns dynamic fusion ratios to combine embeddings from multiple sentences, capturing a broader phonetic range.
Multi-Dimensional Fusion Reward
To ensure high-fidelity synthesis, the agent is guided by a weighted fusion reward $R$ comprising:
- Speaker Similarity: Evaluated using a pre-trained voiceprint model.
- Speech Quality: Predicted via a Mean Opinion Score (MOS) model.
- Intelligibility: Measured by Word Error Rate (WER) from an automatic speech recognition (ASR) system.
Experiments and Results
ASRRL-TTS was evaluated on LibriTTS and VCTK datasets using VITS and Grad-TTS architectures. Experimental results demonstrate significant improvements over traditional methods. In the single-sentence scenario, similarity scores increased by up to 30%, while few-sentence scenarios saw gains of up to 50%. Subjective evaluations showed MOS improvements of up to 0.25.
A key finding is the "20-sentence threshold": ASRRL-TTS significantly outperforms fine-tuning when reference data is below 20 sentences. Beyond this point, traditional fine-tuning becomes more effective as the risk of overfitting diminishes with more data.
Conclusion
ASRRL-TTS provides an agile and robust solution for low-resource speaker adaptation. By shifting the focus from weight fine-tuning to embedding space search, the framework achieves superior speaker similarity without sacrificing audio quality or intelligibility. Future work will explore extending this RL approach to adapt prosody and emotional expression.
Summary updated