A fine pitch model for speech

Jasha Droppo, Alex AceroInterspeech 2007

TL;DR

This paper introduces a "fine pitch model" (FPM) that precisely tracks the fundamental frequency (pitch) of speech on a sample-by-sample basis, overcoming limitations of traditional frame-based methods. This finer-grained modeling allows for more accurate representation of voiced speech, particularly in challenging acoustic conditions.

What Problem Does This Paper Address?

Accurate modeling of speech structure is crucial for many audio processing tasks like enhancement, synthesis, and recognition. Traditional methods often rely on a "mixed excitation model" where speech is seen as a combination of a vocal tract model and an excitation signal. For voiced speech, this excitation is typically modeled as a sequence of impulses with a constant fundamental frequency (pitch) within short analysis frames (25-45ms). However, this assumption is problematic because:

  1. Pitch isn't constant within a frame: The fundamental frequency of human speech naturally changes even within these short frames.
  2. Pitch is event-driven, not continuous: Pitch is a consequence of discrete glottal closure events, not a smoothly varying parameter.

These limitations mean that standard harmonic models can fail to accurately capture the energy in voiced speech, especially when pitch is changing, leading to imperfect residuals and reduced performance in downstream applications.

Methods

The paper proposes a fine pitch model (FPM) that operates time-synchronously with the digitized speech samples, assigning a unique pitch estimate for every sample. This contrasts with frame-based methods that infer a single pitch for an entire segment.

The core idea is to model the instantaneous pitch period ($T_n$) as a sequence that is piecewise constant, reflecting the discrete nature of glottal events. The FPM aims to find a sequence of pitch period estimates that minimizes an objective function. This function balances two terms:

  1. Residual Energy: How well the current speech sample ($y[n]$) can be predicted by a past sample ($y[n-T_n]$) using a time-varying comb filter. Minimizing this term aims to remove the voiced speech components.
  2. Smoothness of Pitch Sequence: A penalty term that encourages the pitch estimates ($T_n$) to be smooth over time, specifically using a first-order Markov assumption ($T_n$ depends on $T_{n-1}$).

This objective function is optimized using standard dynamic programming techniques, involving a forward and a backward recursion to find the optimal sequence of instantaneous pitch periods.

Datasets

Not discussed in the paper. The paper uses simulated speech signals (Figures 3 and 5) and real speech signals (Figures 2, 4, and 6) for demonstration and evaluation, but does not refer to specific named datasets.

Key Insights and Ideas

  • Pitch is an event, not a smooth parameter: The fundamental insight is that pitch is a byproduct of discrete glottal events, leading to a piecewise constant sequence of pitch periods, not a continuously varying one.
  • Frame-based pitch estimation is inherently limited: By assuming a constant pitch within a frame, traditional methods miss crucial dynamic information, leading to imperfect modeling of voiced speech.
  • Sample-synchronous modeling offers higher fidelity: Tracking pitch at every sample allows for a much more precise representation of the voiced speech signal, capturing nuances missed by frame-based approaches.
  • The "stair-step" pattern of pitch: The FPM naturally produces a pitch contour that resembles a stair-step, reflecting the discrete nature of glottal closures.

Key Results

  • The FPM significantly reduces the residual energy for voiced speech, especially when pitch is changing, compared to traditional harmonic models (demonstrated in Figures 3 vs. 5 and Figure 6). This indicates a more accurate capture of the speech signal's energy.
  • The model is shown to work well even in low signal-to-noise environments.
  • The paper discusses the required precision for pitch estimation to accurately place harmonics, highlighting that the FPM's sample-synchronous approach is well-suited to meet these demands.

Limitations and Future Work

  • Computational Complexity: While dynamic programming is used, the paper notes that the first-order Markov assumption for pitch smoothness is chosen for computational efficiency. More complex models might offer higher accuracy but at a greater computational cost.
  • Two-Pass Estimation: The current system uses a two-pass approach: first estimating pitch to within one sample, and then refining it with a limited variation. The paper suggests this is a practical implementation choice, implying that a single-pass, higher-precision method might be desirable.
  • Future Work: The authors imply that further exploration of how to define "substantially similar" segments and how to answer questions about segment definition could lead to different implementations of the FPM with varying accuracy and efficiency. The paper also suggests that the FPM's ability to precisely track pitch could be beneficial for various speech processing applications.

Summary updated

BibTeX
@inproceedings{droppo2007fine,
  author = {Jasha Droppo and Alex Acero},
  title = {{A fine pitch model for speech}},
  booktitle = {Interspeech},
  year = {2007},
}
Figure