Introduction
Emotional Voice Conversion (EVC) is the task of altering the emotional state of a voice recording—for example, transforming a neutral statement into an angry one—while preserving the original linguistic content and the unique identity of the speaker.
A primary challenge in current EVC research is speaker dependency. Most existing models are tailored to specific individuals, requiring extensive data from a single speaker to learn their unique emotional inflections. This limitation prevents models from scaling to "unseen" speakers. Additionally, emotion is hierarchical, affecting both the spectrum (timbre) and prosody (pitch and rhythm). Capturing these multi-scale temporal dependencies while maintaining speaker identity remains a significant technical hurdle.
This paper introduces a speaker-independent EVC framework designed to "convert anyone's emotion" without requiring parallel training data or prior knowledge of the target speaker.
Methods: The VAW-GAN Framework
The authors propose a framework based on the Variational Autoencoding Wasserstein Generative Adversarial Network (VAW-GAN). Their approach is built upon three technical pillars:
- Dual-Pipeline Architecture: Instead of a monolithic model, the system employs two separate VAW-GAN pipelines: one dedicated to spectral features and another for prosody (F0). This specialization allows the model to better capture the distinct characteristics of timbre versus pitch.
- Continuous Wavelet Transform (CWT) for Prosody: Rather than simple linear pitch transformations, this framework uses CWT to decompose the F0 contour into multiple time scales. This captures everything from fine-grained "micro-prosody" to sentence-level intonation, allowing for more natural emotional melodies.
- F0 Conditioning for Disentanglement: To ensure the encoder learns an emotion-independent representation (latent code $z$), F0 features are fed directly into the spectral decoder. This forces the encoder to strip prosodic information from the latent code, effectively disentangling "what is said" from "how it is being said."
Experimental Setup
To evaluate the system's ability to generalize, the authors combined three corpora: an English emotional speech corpus, EmoV-DB, and the JL-Corpus. Crucially, two speakers from the JL-Corpus were reserved as unseen speakers—individuals the model never encountered during training—to test the system's "universal" conversion capabilities.
Key Results
The proposed model, CWT-C-VAWGAN, was evaluated against several baselines using both objective metrics and subjective listening tests.
- Objective Performance: The model achieved lower Mel-cepstral distortion (MCD) and higher Pearson Correlation Coefficients (PCC) for pitch contours compared to baselines. The performance gap between seen and unseen speakers was remarkably small, demonstrating the model's robustness.
- Subjective Quality: In Mean Opinion Score (MOS) tests, the proposed model scored 2.808, outperforming the baseline that lacked F0 conditioning.
- Emotion Similarity: In preference tests, the speaker-independent model actually outperformed speaker-dependent models for unseen speakers. This suggests that training on a diverse range of voices helps the model learn a more robust, "universal" representation of emotion.
Insights and Future Work
The study validates the hypothesis that humans share common acoustic cues for emotion regardless of individual identity. By conditioning the decoder on F0, the latent space becomes cleaner, preventing the original emotion from "leaking" into the output.
While successful, the authors identify several areas for future research. The current study focused primarily on the transition from "neutral" to "angry"; future work will explore more complex emotions like fear or disgust. Additionally, replacing the WORLD vocoder with modern neural vocoders (such as HiFi-GAN) could further improve the naturalness and clarity of the synthesized speech.
Summary updated