Glottal-based analysis of the lombard effect

Thomas Drugman, Thierry DutoitInterspeech 2010

TL;DR

This paper investigates how the human voice source (glottis) changes when speakers talk in noisy environments (the Lombard effect). By analyzing glottal flow features, the authors found significant modifications in vocal effort, which could improve speech processing systems like recognition and synthesis.

What Problem Does This Paper Address?

When people speak in noisy environments, their speech characteristics change to improve intelligibility – this is known as the Lombard effect. While previous research has focused on how the vocal tract (e.g., formant frequencies, spectral tilt) and prosody (e.g., pitch, intensity) change, the behavior of the vocal folds themselves during the Lombard effect has been less explored. This paper aims to fill that gap by analyzing the glottal source, which is the fundamental engine of voice production, to understand its role in Lombard speech.

Methods

The core of this research lies in estimating and analyzing the glottal flow. The authors employ a closed-phase analysis to extract the glottal flow from the speech waveform. This involves:

  • Glottal Closure and Opening Instant (GCI/GOI) Detection: An algorithm from [12] is used to accurately locate these crucial points in the glottal cycle.
  • Glottal Flow Estimation: For voiced regions, a Discrete All-Pole (DAP) model is applied during the estimated closed phase of the glottis. This model, which uses the Itakura-Saito distance for parameter estimation, aims to capture the vocal tract's response. The glottal flow is then obtained through an inverse filtering step.
  • Feature Extraction: Several features are extracted from the estimated glottal flow:
  • Normalized Amplitude Quotient (NAQ): Describes the glottal closing phase.
  • Quasi-Open Quotient (QOQ): Measures the relative open time of the glottis.
  • H1-H2 Ratio: The ratio of amplitudes of the first two harmonics of the glottal source spectrum, indicative of voice quality.
  • Harmonic Richness Factor (HRF): Quantifies the number of harmonics in the glottal spectrum.
  • Fundamental Frequency (F0): A standard measure of pitch.
  • Spectral Ratios (E2-1, E3-1): Ratios of energy in different frequency bands of an averaged speech spectrum, used to capture spectral tilt.

Datasets

The study utilizes a database specifically designed for robust speech recognition. It comprises recordings from 25 speakers (11 female, 14 male). The data includes:

  • Clean speech: Approximately 350 phonetically balanced sentences and 57 sequences of words and numbers.
  • Lombard speech: Recorded in noisy conditions with four types of noise (car, crowd, factory, pop music) at two levels (76 dB-SPL and 84 dB-SPL). For noisy conditions, only the 57 sequences of words and numbers were used.

The speech signals were captured by a close-talk microphone and sampled at 16kHz.

Key Insights and Ideas

  • Glottal Source is Significantly Modified: Contrary to the focus on vocal tract changes, this paper demonstrates that the glottal source itself undergoes substantial modifications during Lombard speech. This suggests a more holistic approach is needed to understand and compensate for the Lombard effect.
  • Increased Vocal Effort is Key: The observed changes in glottal features are attributed to increased vocal effort when speaking in noise. This is a direct physiological response to enhance speech production.
  • Spectral Tilt Changes: The study confirms previous findings that Lombard speech has more high-frequency energy. This is reflected in the glottal source by a richer harmonic content (increased HRF) and a reduced amplitude difference between the first two harmonics (decreased H1-H2).
  • Time-Domain Features Reflect Glottal Dynamics: Features like NAQ and QOQ, while harder to interpret intuitively, show clear differences, particularly in the rapidity of the glottal open phase, suggesting altered glottal adduction/abduction dynamics.
  • Noise Type and Level Matter: The degree of glottal modification varies with noise type and level, with music, crowd, car, and factory noises showing progressively stronger effects.

Key Results

  • Fundamental Frequency (F0): A clear increase in F0 is observed as noise levels increase, corroborating existing literature. For instance, with factory noise at 84 dB, F0 increased by up to 25.7%.
  • NAQ and QOQ: These time-domain features show a significant drop in Lombard speech. For factory noise at 84 dB, NAQ decreased by up to 26.4%, and QOQ by up to 12.6%.
  • H1-H2 Ratio: This spectral feature also decreases, indicating a relative increase in higher harmonics.
  • HRF: The Harmonic Richness Factor increases, signifying more harmonic content in the glottal spectrum.
  • Spectral Energy Ratios (E2-1, E3-1): These ratios confirm an increase in high-frequency energy in Lombard speech, with E2-1 increasing by up to 12.8% for factory noise at 84 dB.

Limitations and Future Work

Limitations:

  • The paper notes that while all speakers tend to modify their glottal source similarly, the degree of change varies among individuals.
  • Interpreting the time-domain features (NAQ, QOQ) intuitively is challenging, though their impact on the glottal waveform is illustrated.

Future Work:

  • The authors suggest that these findings on glottal modifications can be integrated into speech recognition and speaker recognition systems for improved performance.
  • They also propose that these features could be used to enhance parametric speech synthesizers based on source-filter models, potentially improving intelligibility by adapting voice quality.

Summary updated

BibTeX
@inproceedings{drugman2010glottal,
  author = {Thomas Drugman and Thierry Dutoit},
  title = {{Glottal-based analysis of the lombard effect}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure