TL;DR
This paper delves into the acoustic features that make mimicry speech convincing, analyzing suprasegmental, segmental, and subsegmental levels. They found that while imitators can get close to target pitch and duration, spectral features related to vocal tract shape are much harder to replicate, suggesting that perceptual mimicry relies heavily on prosodic and excitation characteristics.
What Problem Does This Paper Address?
Mimicry, or voice imitation, is a fascinating human ability that has seen limited academic study, partly due to the scarcity of standardized datasets and professional imitators willing to participate. This paper aims to dissect what makes a mimicry speech convincing by analyzing the acoustic features of imitated speech compared to both the original target speaker and the imitator's natural voice. The core challenge is to understand which acoustic cues are most critical for a listener to perceive a speech utterance as a successful imitation.
Methods
The researchers employed a multi-level analysis of mimicry speech, categorizing features into three levels:
- Suprasegmental Features: These capture the broader speaking habits and prosody. The paper focuses on pitch contour (fundamental frequency, F0) and duration. Pitch contour analysis involved extracting F0 using Zero Frequency Filtering and comparing the contours of target (T), imitation (I), and natural (N) voices. Duration analysis looked at global duration values.
- Segmental Features: These represent the more static characteristics of the vocal tract shape. Linear Prediction Coefficients (LPC) were extracted, and the Itakura distance was used to measure the spectral similarity between T vs. I and I vs. N.
- Subsegmental Features: These are very short-duration features related to the excitation source. The paper analyzed Strength of Excitation (SoE), derived from the glottal activity, and a measure of perceived loudness (η), based on the abruptness of glottal closure.
Perceptual evaluations were crucial for selecting "best" and "poorly" imitated utterances for detailed acoustic analysis.
Datasets
The paper describes a custom dataset created for this study:
- Target Voices: Recordings of five popular Indian celebrities (PO, MB, PR, NG, SP) from interviews and movies.
- Imitated Voices: A professional mimicry artist imitated ten utterances from each celebrity five times.
- Natural Voices: The imitator also recorded the same utterances in his own natural voice.
All recordings were made in a studio at a sampling frequency of 48 kHz.
Key Insights and Ideas
- Prosody is King (for perception): The study strongly suggests that suprasegmental features like pitch contour and duration are key to successful mimicry. When an imitator successfully matches the target's pitch contour and timing, listeners are more likely to perceive it as a good imitation, even if other features aren't perfectly matched.
- Vocal Tract Shape is Hard to Mimic: Segmental features, which are directly tied to the physical shape of the vocal tract, proved difficult for the imitator to replicate. The Itakura distance was consistently higher between the imitation and the target than between the imitation and the imitator's natural voice, indicating that spectral characteristics are not easily altered.
- Excitation Matters for Subtleties: Subsegmental features like Strength of Excitation (SoE) and loudness (η) also play a role. The imitator showed an effort to adjust these features to match the target, particularly in well-imitated cases, suggesting they contribute to the finer details of perceived vocal quality.
- Best vs. Poor Imitations Differ Significantly: The analysis clearly showed distinct patterns for well-imitated versus poorly-imitated utterances, particularly in pitch contour and subsegmental features. This highlights that the imitator's success varies across different acoustic dimensions.
Key Results
- Perceptual Evaluation: Two evaluations were conducted. The first, with 10 listeners, yielded mean scores indicating good overall mimicry quality, with celebrities PO and MB being imitated particularly well. The second, a blind evaluation with 30 listeners, confirmed that the imitator was often able to fool listeners into identifying the target speaker, with 21 out of 30 imitated utterances correctly identified.
- Pitch Contour: Figures 1 and 2 visually demonstrate that the pitch contours of "best" imitated utterances (Figure 1) show a much closer match between imitation (I) and target (T) than poorly imitated ones (Figure 2). Table 3 shows lower deviation between I and T for best imitations.
- Duration: Table 4 shows that the imitator's global duration values were generally closer to the target than to his natural voice, though not always perfectly matched.
- Itakura Distance: Table 5 consistently shows a higher Itakura distance between Imitation (I) and Target (T) compared to Imitation (I) and Natural (N) for all celebrities, indicating that spectral features are not well-matched.
- Strength of Excitation (SoE) & Loudness (η): Scatter plots (Figures 3 & 4) suggest that the imitator attempts to adjust SoE to match the target, especially in good imitations. Table 6 shows that loudness measures (η) for PO, MB, and NG tend to move towards the target values.
Limitations and Future Work
- Limitations: The paper acknowledges that the spectral features (segmental level) are difficult to manipulate, and the Itakura distance analysis confirms this. The study also notes that while the imitator can match global timing, local timing might be less precise. The limited number of target speakers and utterances, while extensive for a custom study, could be expanded.
- Future Work: The authors suggest that further research could explore the interaction between different feature levels and their combined effect on perception. Investigating how different types of mimicry (e.g., caricature vs. exact imitation) affect these features would also be valuable. The development of more comprehensive and standardized mimicry speech databases is also a clear need.
Summary updated