TL;DR
This paper investigates how biases in training data affect the intelligibility and naturalness of synthesized person names generated by a neural text-to-speech (TTS) system. They found that names common in the training data and those of domestic origin are perceived significantly better, highlighting a critical bias that needs addressing for equitable representation in synthesized speech.
What Problem Does This Paper Address?
As text-to-speech (TTS) systems become more sophisticated, their application in real-world scenarios like celebratory ceremonies (graduations, awards) is growing. A crucial aspect of these events is the accurate and natural pronunciation of attendees' names. However, existing TTS systems are often trained on limited datasets, which can lead to biases. This paper tackles the problem of how these biases, particularly those related to name frequency and origin, impact the perception of synthesized person names. The authors aim to understand these biases to eventually develop voice conversion techniques that can improve the rendering of underrepresented names.
Methods
The researchers utilized a pre-trained English TTS system based on Tacotron2 and WaveNet vocoder. They synthesized a set of 24 person names, varying them across three factors: name frequency (high, mid, low, and not in corpus), name length (two or three syllables), and origin (US domestic, international, blended). These synthesized names were then perceptually evaluated by listeners on four criteria: intelligibility, naturalness, ability to reproduce the name in written form, and similarity to original spoken names (for names present in their reference corpus). Additionally, they calculated the Levenshtein distance between the written forms of the original and perceived names to quantify orthographic differences, serving as a proxy for intelligibility and naturalness.
Datasets
- Training Data for TTS System: LJ Speech Dataset (a single-speaker corpus with approximately 24 hours of speech data).
- Reference Corpus for Analysis: A corpus of spoken person names from five years of public commencement ceremonies in computing disciplines at a US technical university. This corpus contained over 2,700 samples of formally read person names.
- Name Components for Synthesis: The CMU Pronouncing Dictionary was used to derive name components for synthesis.
Key Insights and Ideas
- Frequency Matters: Higher frequency names in the training data are perceived with significantly better intelligibility and naturalness. This suggests that TTS systems learn to pronounce common names more effectively.
- Domestic Bias is Strong: Names of US domestic origin are rated substantially higher in intelligibility and naturalness compared to international names. This points to a clear monolingual and cultural bias in the training data.
- Levenshtein Distance as a Proxy: The Levenshtein distance between the written original and perceived name correlates with intelligibility and naturalness. Larger distances indicate poorer perception, especially for international names.
- Underrepresentation is a Challenge: Names not present in the training corpus, and by extension, underrepresented names (like those from minority groups), are likely to be synthesized with lower quality.
Key Results
- High-frequency names were characterized by improved perception for intelligibility over low-frequency names.
- Intelligibility and naturalness ratings for domestic US names were substantially higher than for international names.
- For the perceptual Likert ratings, domestic names received a mean score of 4.8 (std. 0.4) for similarity, while international names received 2.3 (std. 1.4).
- Low-frequency names were characterized by more orthographic distance (higher Levenshtein distance) compared to high-frequency names.
Limitations and Future Work
The authors acknowledge that the synthesized names were not found in full in their reference corpus, and name components were concatenated randomly based on frequency group. This approach might not fully replicate real-world name structures. Future work aims to apply name voice conversion to improve the rendering of international, low-frequency, and underrepresented person names, particularly for high-stakes celebratory contexts.
Summary updated