TL;DR
This paper introduces a new fundamental frequency (F0) estimation method that significantly improves upon existing techniques like DIO. By employing Recursive Moving Average (RMA) filters for harmonic attenuation and a novel period detector, the method achieves state-of-the-art accuracy while being remarkably faster, making it ideal for real-time applications.
What Problem Does This Paper Address?
Estimating the fundamental frequency (F0) of speech is crucial for many audio processing tasks, including speech synthesis, voice transformation, and singing voice analysis. While many F0 estimators exist, there's a persistent trade-off between accuracy and computational speed. Existing methods often struggle with signals containing higher harmonics or low fundamental component amplitudes, and real-time applications demand highly efficient algorithms. The paper aims to bridge this gap by developing an F0 estimator that is both fast enough for real-time use and highly accurate, even in challenging signal conditions.
Methods
The proposed method builds upon the DIO framework but introduces key improvements.
- Recursive Moving Average (RMA) Filters: Instead of traditional FIR filters for harmonic attenuation, the authors use RMA filters. These filters are computationally efficient, especially when implemented with powers of two for N (the filter order), allowing for bit shifts instead of multiplications. A set of six cascaded RMA filters is used per band to achieve sufficient harmonic attenuation.
- Novel Period Detector: A significant contribution is a new period detector designed to reliably estimate F0 even in the presence of higher harmonics and low fundamental component amplitudes. This detector operates by identifying positive/negative zero crossings and peaks. It incorporates rules to ignore less significant peaks and zero crossings within a certain time frame, and it can declare "no period" if no events are detected within a maximum interval. This detector outputs the period (T), detection time (t), and a "Fundamental-period likeliness" (λ) score.
- Candidate Selection: The method uses the "Fundamental-period likeliness" score from multiple bands to select the most suitable F0 candidate. It also incorporates a dynamic programming-based post-processing step (similar to RAPT) that uses F0 continuity and peak amplitude to refine the estimation. The trace-back duration of this post-processing can be adjusted to balance accuracy and latency.
Datasets
The evaluation used publicly available natural speech databases:
- CMU Arctic database: Three speakers (bdl, jmk, slt) were used (DB1-DB3).
- Bagshaw's database: Two speakers (rl, sb) were used (DB4-DB5).
The ground truth F0 and voicing decisions were derived from simultaneously recorded Electroglottography (EGG) signals using the NDF algorithm, which is noted for its high accuracy.
Key Insights and Ideas
- RMA Filters for Efficiency: The shift from FIR to RMA filters for harmonic attenuation is a clever way to leverage computational efficiency, particularly for real-time processing.
- Robust Period Detection: The novel period detector's ability to handle signals with higher harmonics and low fundamental amplitudes is a critical advancement, addressing a known weakness in methods like DIO.
- "Fundamental-period likeliness" as a Key Metric: Using this score for candidate selection and post-processing provides a robust measure of how likely a detected period is to be the true fundamental frequency.
- Post-processing Trade-off: The paper highlights that the dynamic programming post-processing improves accuracy but adds latency. The ability to tune the trace-back duration offers flexibility for different application needs.
- DIO's Weakness: The paper points out that DIO can sometimes mistake low-frequency noise for fundamental components, leading to errors, especially when combined with its post-processing. The proposed method appears to mitigate this.
Key Results
- Accuracy: The proposed method achieved better accuracy than SWIPE', DIO, and YIN when a trace-back duration of 30 ms was applied. It was also more accurate than DIO and YIN even without post-processing. While NDF generally performed slightly better in terms of Gross Error Rate (GER) and Root Mean Square Error (RMSE), the proposed method was a strong contender.
- Speed: The proposed method was significantly faster than the compared methods. It was approximately 30 times faster than SWIPE' and considerably faster than DIO. The post-processing added only a small overhead to the processing time.
- Error Distribution: Kernel density estimation showed that the proposed method and NDF had estimation errors concentrated well around 0 Hz, indicating high precision.
Limitations and Future Work
- Limitations: The paper acknowledges that NDF generally achieved slightly better accuracy. While the proposed method is fast and accurate, it's noted that its accuracy is still slightly worse than NDF. The authors also mention that the post-processing introduces latency, which might be a concern for strict real-time applications if a long trace-back duration is used.
- Future Work: The authors plan to evaluate and improve the method for singing voice applications, where real-time processing is even more prevalent.
Summary updated