A Robust and Low Computational Cost Pitch Estimation Method

Desheng Wang, Yangjie Wei, Yi Wang, Jing WangSensors 2022 · DOI

TL;DR

This paper introduces a novel pitch estimation method that improves accuracy and robustness, especially in noisy conditions, by relaxing the strict harmonic integer multiple assumption and employing an efficient pitch segment extension technique. The proposed method achieves comparable or better accuracy than existing state-of-the-art methods while significantly reducing computational cost, making it suitable for real-time applications.

What Problem Does This Paper Address?

Pitch estimation, a fundamental task in speech and audio processing, aims to determine the fundamental frequency (pitch) of a sound. While widely used in areas like speech interaction, music analysis, and medical diagnosis, existing methods struggle with robustness, particularly in the presence of noise and reverberation. A core challenge lies in the assumption that harmonic frequencies in audio signals are strict integer multiples of the fundamental pitch. In reality, factors like vocal tract acoustics and spectral leakage cause these harmonics to deviate slightly. This deviation, coupled with the computational expense of many robust methods, limits their effectiveness in real-world, resource-constrained scenarios.

Methods

The proposed method builds upon the Summation of Residual Harmonics (SRH) algorithm, introducing two key modifications:

  1. Loose Harmonic Constraint: Instead of enforcing strict integer multiples for harmonic frequencies, the method allows for a small deviation (controlled by a parameter $\Delta f$). This "loose constraint" enables the algorithm to better capture actual harmonic frequencies that might be slightly shifted from their theoretical integer positions. This is visualized in Figure 4, where the "proposed" method captures harmonics that deviate from strict integer multiples.
  2. Low-Cost Pitch Segment Extension: A novel "main pitch segment extension" scheme is introduced. This technique leverages the "smooth prior" of pitch trajectories (the idea that pitch changes gradually over time). It identifies segments of high-confidence pitch estimates and extends them forward and backward to fill gaps and correct short-term errors. This scheme is also integrated into the voiced/unvoiced decision process to further reduce errors at the boundaries of voiced segments.

Datasets

The evaluation utilizes the following datasets:

  • Pitch Dataset: The widely used Keele database, containing phonetically balanced speech from ten speakers.
  • Noise Dataset: The NOISE-X92 dataset, used to add white and four types of colored noise to the clean speech at various Signal-to-Noise Ratios (SNRs).
  • Reverberation Dataset: Clean speech convolved with 24 room impulse responses (RIRs) generated using the image source method, simulating different reverberation levels (quantified by RT60).

No new benchmarks were introduced; existing datasets were used to ensure comparability with other methods.

Key Insights and Ideas

  • Harmonic Deviations are Real and Important: The paper highlights that the assumption of strict integer multiples for harmonics is a significant limitation in current pitch estimation. Allowing for slight deviations is crucial for robustly capturing real-world audio signals.
  • Smoothness is a Powerful Prior: Leveraging the natural smoothness of pitch trajectories is an effective way to improve robustness and reduce errors, especially at the beginning and end of voiced segments.
  • Efficiency Matters: The proposed method demonstrates that achieving robustness doesn't necessarily require high computational complexity. The segment extension technique is designed to be computationally efficient.
  • Voiced/Unvoiced Decisions Benefit from Context: Integrating the pitch segment extension idea into the voiced/unvoiced decision process shows that considering temporal context (beyond just the current frame) significantly improves accuracy.

Key Results

  • Accuracy: The proposed method demonstrates better overall accuracy and robustness compared to ten other pitch estimation methods, particularly under noisy and reverberant conditions.
  • Comparison to State-of-the-Art: It achieves performance comparable to or better than the state-of-the-art fast NLS method, especially at lower SNRs and stricter accuracy requirements (lower $p$ thresholds). It also performs competitively with methods like SWIPE and CREPE.
  • Speed: The proposed method is significantly faster, with a time cost approximately 1/8th of the state-of-the-art fast NLS method on the experimental computer. Its run time ratio ($\Gamma$) is 0.49, meaning its computation time is roughly half the actual audio signal duration.
  • Voiced/Unvoiced Decision Accuracy: The proposed method shows superior voiced/unvoiced decision accuracy compared to the original SRH method, with improvements often exceeding 5% and sometimes reaching 10%.

Limitations and Future Work

  • Long-Term Median Latency: The paper notes that the "long-term median" used in the method can increase output latency. A potential improvement is to replace it with a "short-term median" that updates over time, reducing latency for subsequent pitch estimates.
  • Further Segment Extension Applications: The authors are investigating using the segment extension approach to further improve voiced/unvoiced decisions by expanding them, not just "narrowing" them.
  • Parameter Tuning: While $\Delta f$ and $f_{shift}$ are discussed, optimal parameter tuning for diverse acoustic conditions might still be an area for exploration.

Summary updated

Abstract

Pitch estimation is widely used in speech and audio signal processing. However, the current methods of modeling harmonic structure used for pitch estimation cannot always match the harmonic distribution of actual signals. Due to the structure of vocal tract, the acoustic nature of musical equipment, and the spectrum leakage issue, speech and audio signals' harmonic frequencies often slightly deviate from the integer multiple of the pitch. This paper starts with the summation of residual harmonics (SRH) method and makes two main modifications. First, the spectral peak position constraint of strict integer multiple is modified to allow slight deviation, which benefits capturing harmonics. Second, a main pitch segment extension scheme with low computational cost feature is proposed to utilize the smooth prior of pitch more efficiently. Besides, the pitch segment extension scheme is also integrated into the SRH method's voiced/unvoiced decision to reduce short-term errors. Accuracy comparison experiments with ten pitch estimation methods show that the proposed method has better overall accuracy and robustness. Time cost experiments show that the time cost of the proposed method reduces to around 1/8 of the state-of-the-art fast NLS method on the experimental computer.
BibTeX
@inproceedings{wang2022robust,
  author = {Desheng Wang and Yangjie Wei and Yi Wang and Jing Wang},
  title = {{A Robust and Low Computational Cost Pitch Estimation Method}},
  booktitle = {Sensors},
  year = {2022},
  doi = {10.3390/s22166026},
}
Figure