On the effect of fundamental frequency on amplitude and frequency modulation patterns in speech resonances

Pirros Tsiakoulis, Alexandros PotamianosInterspeech 2010 · OpenReview

Abstract

Amplitude modulation (AM) and frequency modulation (FM) in speech signals are believed to reflect various non-linear phenomena during the speech production process. In this paper, the amplitude and frequency modulation patterns are analyzed for the first three speech resonances in relation to the fundamental frequency ($F_0$). The formant tracks are estimated, and the resonant signals are extracted and demodulated. The Amplitude Modulation Index (AMI) and Frequency Modulation Index (FMI) are computed, and examined in relation to the $F_0$ value, as well as the relation between $F_0$ and the first formant value ($F_1$). Both AMI and FMI are significantly affected by pitch, with modulations being more frequently present in low $F_0$ conditions. Evidence of non-linear interaction between the glottal source and the vocal tract is found in the dependence of the modulation patterns on the ratio of $F_1$ over $F_0$. AMI is amplified when pitch harmonics coincide with $F_1$, while FMI shows complementary behavior.

Introduction

Traditional speech processing often relies on a "source-filter" model, which assumes the vocal source (the glottis) and the filter (the vocal tract) operate independently. However, real speech is characterized by significant non-linearities and complex interactions. While AM-FM models—which treat speech as a collection of resonances that modulate in both amplitude and frequency—have improved speech recognition, the underlying physics of these modulations remains poorly understood.

This paper explores how the fundamental frequency ($F_0$, or pitch) influences these non-linear amplitude and frequency modulations (AM-FM) in vocal resonances. The researchers identified that lower pitch leads to more frequent modulations and discovered a "mode-locking" phenomenon where resonances stabilize when aligned with pitch harmonics.

Methodology

The study utilizes an AM-FM analysis framework to decompose speech into its constituent resonances (formants) using the TIMIT database, which provides a diverse range of male and female speakers.

  1. Demodulation: The researchers employed the Teager-Kaiser Energy Operator (TEO) and the Energy Separation Algorithm (ESA) to extract the instantaneous amplitude and frequency of resonant signals.
  2. Amplitude Modulation Index (AMI): The amplitude envelope is modeled using a multi-pulse approach. AMI is defined as the ratio of a "secondary" pulse to the "primary" pulse within a single pitch period.
  3. Frequency Modulation Index (FMI): This metric quantifies the deviation of the instantaneous frequency from the average formant frequency.
  4. Statistical Analysis: The first three formants ($F_1, F_2, F_3$) were tracked and their modulation indices compared against the speaker's $F_0$.
Figure 2: Illustration of the AM-FM analysis framework, showing the speech signal (a), extracted envelopes for F1 and F2 with primary and secondary pulses (b, d), and instantaneous frequency estimates (c, e).

Results and Discussion

The analysis reveals that modulation is not random but is heavily dependent on pitch. As $F_0$ increases, the strength and frequency of these modulations decrease across all formants and for both genders.

Pitch as a Driver of Modulation

Both AMI and FMI show a clear decreasing trend as $F_0$ increases. This suggests that at lower pitches, the vocal folds have more time to achieve complete glottal closure within each cycle, creating stronger secondary excitations that manifest as amplitude modulations. The decrease in average AMI at higher pitches is primarily due to the total disappearance of secondary pulses rather than a gradual reduction in their size.

Figure 4: Average AMI and FMI values plotted against fundamental frequency (F0) for male and female speakers, showing a consistent downward trend as pitch increases.

Source-Filter Interaction and Mode Locking

The data provides strong evidence for non-linear source-filter interaction. A striking finding is the "tuning" between $F_1$ and $F_0$. When the resonance frequency is an exact integer multiple of the pitch (e.g., $F_1/F_0 = 3$), the system enters a state of "mode locking." In these states, AMI peaks while FMI reaches a valley. This indicates that when a resonance aligns with a pitch harmonic, the amplitude modulation is amplified, but the frequency becomes more stable, suggesting the resonance is "locking" onto the harmonic.

Conclusion

This research demonstrates that the fundamental frequency significantly influences the non-linear modulation patterns in speech resonances. While the study is primarily observational, it highlights the importance of source-filter interactions in speech production. Future work should focus on theoretical physical modeling to explain these non-linearities and developing "pitch-aware" modulation features to enhance the robustness of speech technologies like recognition and speaker identification.

Summary updated

Abstract

Amplitude modulation (AM) and frequency modulation (FM) in speech signals are believed to reflect various non-linear phenomena during the speech production process. In this paper, the amplitude and frequency modulation patterns are analyzed for the first three speech resonances in relation to the fundamental frequency (F0). The formant tracks are estimated, and the resonant signals are extracted and demodulated. The Amplitude Modulation Index (AMI) and Frequency Modulation Index (FMI) are computed, and examined in relation to the F0 value, as well as the relation between F0 and the first formant value (F1). Both AMI and FMI are significantly affected by pitch, with modulations being more frequently present in low F0 conditions. Evidence of non-linear interaction between the glottal source and the vocal tract is found in the dependence of the modulation patterns on the ratio of F1 over F0. AMI is amplified when pitch harmonics coincide with F1, while FMI shows complementary behavior.
BibTeX
@inproceedings{tsiakoulis2010effect,
  author = {Pirros Tsiakoulis and Alexandros Potamianos},
  title = {{On the effect of fundamental frequency on amplitude and frequency modulation patterns in speech resonances}},
  booktitle = {Interspeech},
  year = {2010},
  url = {https://openreview.net/forum?id=0YXznUznmT},
}

Loading related content...

Figure