TL;DR
This paper introduces a novel hybrid algorithm for pitch marking in speech signals, combining two existing methods using a finite state machine (FSM). This approach significantly improves the accuracy of identifying pitch marks, crucial for various speech processing applications.
What Problem Does This Paper Address?
Pitch marking, also known as finding "pitch marks" or "epochs," is a fundamental task in speech processing. These marks represent the beginning of a pitch period, essentially the fundamental frequency or "beat" of the voice. Accurate pitch marking is essential for applications like text-to-speech (TTS) synthesis, speech analysis, and more. However, existing methods often struggle with errors like missing pitch marks or falsely detecting them, particularly at vowel boundaries or in noisy segments. This paper aims to overcome these limitations by developing a more robust and accurate pitch marking algorithm.
Methods
The core of this work is a hybrid pitch marking algorithm that leverages the strengths of two individual speech signal-based pitch marking algorithms:
- Complex Harmonic Filter Analysis (CHFA) (PMA1): This method detects the fundamental frequency (F0) contour by analyzing amplitude and phase in complex filters. It uses the phase periodicity for pitch detection and amplitude for voiced/unvoiced decisions.
- Glottal Closure Instant Detection Algorithm (GCIDA) (PMA2): This algorithm uses a multi-channel approach combining short-time analysis with wavelet transforms to pinpoint Glottal Closure Instants (GCIs), which correspond to the most negative peaks in the speech signal.
The proposed hybrid approach involves several stages:
- Preprocessing: A simple moving average filter smooths the speech signal to remove noise and voiceless segments. An energy threshold is applied to discard pitch marks in low-energy regions.
- Alignment Module: Individual pitch marks from PMA1 and PMA2 are aligned to the nearest negative peak (local minimum) within a small search window (±1 msec). Crucially, a confidence score is calculated for each aligned pitch mark. Two methods for calculating this score are explored:
- Frequency of Occurrence: Based on how often a pitch mark appears across combined algorithms.
- Distance to Nearest Negative Point: Measures how close the detected pitch mark is to the actual negative peak.
- Selection Module: This is where the novelty lies. A Finite State Machine (FSM) is used to combine the outputs of PMA1 and PMA2. The FSM's states represent segments between pitch marks, and transitions are weighted by the confidence scores (specifically, the distance-based confidence score). A dynamic programming algorithm finds the "best path" through the FSM, effectively selecting the most reliable pitch marks.
- Postprocessing: Similar to preprocessing, an energy threshold is used to remove pitch marks that might have been shifted to incorrect positions. Additionally, if two pitch marks are too close (less than 2 msec apart), the one with the lower signal amplitude is removed.
Datasets
The paper utilizes two primary datasets for evaluation:
- TC-STAR Database: A subset of this database, consisting of 10 minutes of speech (5 minutes female, 5 minutes male) from high-quality UK English voices, was used for internal evaluation. These pitch marks were initially automatically detected and then manually corrected for high accuracy.
- LMU Database (from ECESS): This database, containing manually marked pitch marks from 44 speakers (22 male, 22 female) of English and German, was used for a gender-dependent evaluation campaign. This dataset is noted for its variety of speakers.
Key Insights and Ideas
- The Power of Hybridization: Combining multiple algorithms, even those based on similar signal types (speech signal in this case), can yield better results than individual methods.
- Finite State Machines for Robustness: Using an FSM to model the relationships and confidence scores between pitch marks from different algorithms provides a structured and effective way to select the most accurate ones. The FSM acts as a sophisticated decision-maker.
- Confidence Scores are Key: The effectiveness of the FSM heavily relies on the quality of the confidence scores. The paper highlights that using the distance to the nearest negative peak as a confidence measure is particularly effective.
- Alignment to Local Minima: Shifting detected pitch marks to the nearest negative peak (local minimum) is a crucial step for improving accuracy, as these peaks often correspond to the actual excitation points in the speech signal.
Key Results
The paper presents quantitative results demonstrating the superiority of the proposed hybrid FSM-based algorithm:
- TC-STAR Database Evaluation:
- The hybrid FSM algorithm achieved a Success Rate (SR%) of 96.50% and an Accuracy (ACC%) of 85.69%.
- This significantly outperformed the individual PMA1 (CHFA) and PMA2 (GCIDA) algorithms, as well as a "Hybrid (Probability)" approach that used a different confidence score.
- ECESS Evaluation (LMU Database):
- The "Hybrid (FSM)" algorithm achieved the best overall results for both male and female speakers, with an SR of 84.48% and an ACC of 67.41%.
- The results generally showed better performance on female speech across most algorithms tested.
Limitations and Future Work
- Performance Discrepancy: While the hybrid FSM algorithm performs well, the paper notes that improvements are still needed, particularly for male speech in the ECESS evaluation.
- Future Directions: The authors suggest that further improvements could be achieved by combining more than two single pitch marking algorithms, especially those based on fundamentally different detection concepts (e.g., incorporating EGG signal-based methods alongside speech signal-based ones). This would create an even more comprehensive and robust system.
Summary updated