Analysis and synthesis of shouted speech

Tuomo Raitio, Antti Suni, Jouni Pohjalainen, Manu Airaksinen, Martti Vainio, Paavo AlkuInterspeech 2013

TL;DR

This paper tackles the challenge of synthesizing "shouted speech," a distinct vocal style often used for emphasis or to overcome noise. By analyzing the acoustic differences between normal and shouted speech and experimenting with various synthesis techniques, the authors demonstrate that while quality degrades, the impression of shouting and vocal effort can be effectively preserved.

What Problem Does This Paper Address?

Communicating effectively in noisy environments or conveying strong emotions often requires vocal styles beyond normal speech. Shouting, in particular, is a distinct vocalization characterized by increased loudness, fundamental frequency (F0), and altered spectral properties. Synthesizing this style is difficult because standard speech synthesis models, trained on normal speech, struggle to capture these extreme acoustic changes. The sparse harmonics in shouted speech can also lead to estimation errors in spectral analysis, further complicating synthesis. Existing research has largely focused on increased vocal effort or Lombard speech, leaving a gap in the explicit synthesis of genuinely shouted speech. This paper aims to bridge that gap by analyzing shouted speech and exploring methods for its artificial generation.

Methods

The authors employed a multi-pronged approach:

  • Analysis: They analyzed acoustic features of both normal and shouted speech, focusing on duration, Sound Pressure Level (SPL), F0, spectral tilt (H1-H2), and glottal pulse shape (NAQ).
  • Spectral Estimation: To address the challenges posed by sparse harmonics in shouted speech, they experimented with different spectral estimation methods. This included conventional Linear Prediction (LP) and Weighted Linear Prediction (WLP) variants like Stabilized WLP with STE (Short-Time Energy) window (STE-WLP) and AME-WLP (Attenuation of the Main Excitation).
  • Synthesis Techniques:
    • Analysis-Synthesis: They used the GlottHMM vocoder with different spectral estimation methods to synthesize speech and evaluated the quality.
  • Voice Conversion: A simple voice conversion method was implemented to adapt synthesized normal speech to a shouted style, leveraging a pulse library from the GlottHMM vocoder.
  • HMM-based Synthesis (Adaptation): They adapted pre-trained HMM-based voices (trained on normal speech) to the shouted style using a small amount of shouted speech data. This involved using techniques like CSMAPLR + MAP, which have been successful for Lombard speech synthesis.

Datasets

  • Primary Recording: 11 male and 11 female native Finnish speakers recorded 24 sentences each, both in normal phonation and shouting. Sentences varied in length and mood (imperative and indicative).
  • Additional Data for Voice Conversion: Shouted speech from two additional speakers whose normal speech databases were already available was recorded. These existing databases contained 599 (male) and 1319 (female) sentences. The two speakers also shouted 100 new sentences each, including those with varying focus conditions and emotional content.

Key Insights and Ideas

  • Shouting is More Than Just Loudness: The analysis reveals significant differences beyond just increased SPL, including higher F0, altered spectral tilt, and changes in glottal pulse shape, all contributing to a distinct vocal quality.
  • Spectral Estimation Matters: Conventional LP struggles with the sparse harmonics of shouted speech, leading to synthesis artifacts. Weighted LP methods, particularly STE-WLP, proved more effective for capturing the spectral characteristics of shouting.
  • Adaptation vs. Voice Conversion Trade-offs: While HMM-based adaptation of normal speech to shouting resulted in some quality degradation (especially in prosody), it preserved the impression of shouting and vocal effort well. Simple voice conversion offered more consistent prosody but less convincing shouting characteristics.
  • Subjectivity is Key: The subjective evaluations highlight that even with quality degradation, the impression of shouting and the perceived vocal effort are crucial for listeners to identify the intended vocal style.

Key Results

  • Acoustic Differences: Shouted speech was significantly longer (20-24%), had higher SPL (17.6-19.6 dB higher), and much higher F0 (71-152% higher) than normal speech. H1-H2 was lower for females (indicating less spectral tilt) but slightly higher for males. NAQ was lower for both genders, suggesting tenser phonation.
  • Synthesis Quality:
    • Analysis-synthesis experiments showed that stabilized STE-WLP performed best for shouted speech synthesis, outperforming conventional LP.
  • Subjective evaluation of HMM-based adapted shouting voices indicated a quality degradation compared to natural shouted speech, but the impression of shouting and vocal effort was well preserved.
  • The simple voice conversion method, while having more consistent prosody, was less effective at conveying the impression of shouting and vocal effort compared to the adapted HMM voices.
  • Preference: In a comparison test, stabilized STE-WLP was preferred over LP, especially for high-pitched female voices.

Limitations and Future Work

  • Quality Degradation: The authors acknowledge that adapted shouting synthesis is rated inferior in quality compared to natural shouted speech. This is attributed to the large acoustic differences between the styles and the limited adaptation data.
  • Prosody Issues: The prosody of the synthesized shouted speech was identified as a particular area of degradation.
  • Future Work: The authors suggest that stepwise adaptation from normal to shouted speech might improve synthesis quality. Further research could also explore more sophisticated voice conversion techniques.

Summary updated

BibTeX
@inproceedings{raitio2013analysis,
  author = {Tuomo Raitio and Antti Suni and Jouni Pohjalainen and Manu Airaksinen and Martti Vainio and Paavo Alku},
  title = {{Analysis and synthesis of shouted speech}},
  booktitle = {Interspeech},
  year = {2013},
}
Figure