TL;DR
This paper delves into the acoustic nuances of voice imitation, revealing how professional impersonators meticulously adjust their vocal tract, pitch, and glottal source characteristics to mimic a target speaker. The findings offer valuable insights into the perception of speaker individuality and the mechanics of voice conversion.
What Problem Does This Paper Address?
Voice impersonation is a fascinating human ability that has long intrigued speech scientists. While we can readily perceive when a voice is being imitated, the precise acoustic mechanisms behind this feat remain largely unexplored. How do impersonators transform their natural voice to convincingly replicate another's? What specific acoustic features do they manipulate, and to what extent? Understanding these questions is crucial for advancing our knowledge of speaker perception, developing more natural synthetic speech, and potentially informing voice conversion technologies. This paper aims to shed light on these mysteries by conducting a detailed acoustic analysis of a professional impersonator's imitated voice compared to both the target speaker and the impersonator's own natural voice.
Methods
The researchers employed a multi-faceted acoustic analysis approach. Key methods included:
- Pitch Frequency Analysis: Examining both the mean pitch and the dynamic contours of the fundamental frequency over time. This was done using the "Pitch Contour" function in WaveSurfer, with parameters like ESPS, a 7.5 ms frame length, and a 10 ms frame period.
- Discrete Fourier Transform (DFT) Spectra: Analyzing the spectral shape of vowels to understand how the overall frequency content of the voice is altered. This involved using overlapping Hanning window frames (64 ms length, 8 ms period) and extracting spectral envelopes from 40-order DFT cepstra.
- Formant Frequency Analysis: Identifying the resonant frequencies of the vocal tract (F1, F2, F3, F4) by peak-picking from the spectral envelopes. Formant frequencies are critical for vowel quality and are strongly influenced by vocal tract shape.
- Glottal Source Characteristics (H1-H2): Measuring the difference in amplitude between the first and second harmonics (H1-H2) from the DFT spectra. This metric provides insight into the vibration patterns of the vocal folds, a key component of voice production.
- Syllable Duration: Manually delimiting syllable boundaries from spectrograms and calculating their durations to assess temporal aspects of speech.
The analysis focused on comparing these acoustic features across the target speaker, the impersonator's imitated voice, and the impersonator's natural voice.
Datasets
The study utilized speech data recorded from two speakers:
- Speaker A: The target speaker, described as a professional comic storyteller.
- Speaker B: The professional impersonator.
Both speakers recorded two specific Japanese sentences:
- "Ichidode i:kara mitemitai, nyo:boga hesokuri kakusutoko." (Just once, I wish to catch my wife hiding her secret savings.)
- "Dekakeru nekoni yukisaki kikeba, ryoko:ga sukide mata tabida." (Asked where to go, the cat replied that he is going on a travel again because he loves to.)
Speaker B recorded each sentence twice: once in his imitated voice (following Speaker A's utterances) and once in his natural voice, on separate days. The data was sampled at 16 kHz with 16-bit resolution. The paper used one sample per sentence for the analysis, resulting in three utterances analyzed per sentence.
Key Insights and Ideas
- Holistic Imitation: Impersonators don't just tweak one aspect of their voice; they orchestrate changes across pitch, vocal tract acoustics, and glottal source characteristics to achieve a convincing imitation.
- Pitch Dynamics Matter: Beyond just the average pitch, the way the pitch changes over time (its contour and tilt) is a significant factor that impersonators seem to replicate. This suggests that the dynamic aspects of speech are crucial for speaker identification.
- Vocal Tract Shape Shifting: The similarity in spectral shapes and formant frequencies (with some exceptions) between the imitated and target voices strongly indicates that impersonators actively alter their vocal tract configuration.
- Glottal Source Mimicry: The close match in H1-H2 values between the imitated and target voices suggests that impersonators can also manipulate the vibration of their vocal folds to match the target speaker's glottal source characteristics, even if it differs significantly from their own natural voice.
- The Mystery of F2: The study noted that while many formant frequencies were closely matched, F2 showed larger differences in some cases. This raises questions about the perceptual salience of F2 for speaker identification or potential difficulties in its precise manipulation by the impersonator.
Key Results
- Pitch: The imitated voice's mean pitch frequency was approximately 20 Hz higher than the target speaker's, suggesting an exaggeration of the target's high-pitched voice. The pitch contours of the imitated voice were more similar to the target speaker's than to the impersonator's natural voice.
- Spectra: The DFT spectra of imitated vowels closely resembled the target speaker's, particularly in spectral peaks and dips. This differed significantly from the impersonator's natural voice spectra.
- Formants: Formant frequency differences between the imitated and target voices were generally within 4%, except for F2. However, differences between the imitated and natural voices were substantial, with some F1 differences reaching up to 62%.
- Glottal Source (H1-H2): The H1-H2 values for the imitated voice were very close to the target speaker's, while being significantly different from the impersonator's natural voice. This suggests successful imitation of the glottal source.
- Syllable Duration: The correlation coefficient between the syllable duration of the target and imitated voices was lower than that between the target and natural voices, indicating that syllable duration might not be as precisely controlled or might be exaggerated by the impersonator.
Limitations and Future Work
The authors acknowledge that their study focused on a single professional impersonator and a specific target speaker. They suggest that future work should:
- Examine the perceptual effects of these acoustic characteristics on speaker individuality.
- Measure morphological differences in the speech production systems of imitated vs. natural phonation.
- Investigate the role of other acoustic features, such as spectral dips, in imitation.
- Explore the potential difficulty in adjusting F2 and its perceptual impact.
Summary updated