TL;DR
This paper introduces a novel "concurrent curve strategy" for automatically tracking formants in speech. By making formant tracks influence each other during deformation, the authors achieve more robust and consistent tracking, simplifying the overall control strategy and improving performance, especially for challenging sounds like nasal vowels.
What Problem Does This Paper Address?
Formants are crucial acoustic features that reveal information about vocal tract shape and are used in applications ranging from speech synthesis to recognition. Automatically tracking these formants in speech signals, however, remains a challenging problem. Previous methods often treated each formant independently, requiring complex control strategies to ensure consistency. A key difficulty arises when formants are close in frequency or when spectral peaks are weak, leading to tracks merging or becoming inaccurate. This paper aims to address this by incorporating the interdependency between formants directly into the tracking process.
Methods
The core contribution is a modification of an existing active curve-based formant tracking method. Previously, each formant track was deformed independently based on spectrogram energy and smoothness constraints. This paper proposes two strategies to introduce interdependency:
- Spectrogram Partition Strategy: The spectrogram is divided into exclusive regions, each assigned to a specific formant. Formant deformation is then guided by the energy within its assigned region. The regions themselves are dynamically updated based on formant track positions.
- Repulsive Tracks Strategy: A "repulsion force" is added to the energy functional that each formant track minimizes. This force pushes neighboring formant tracks away from each other, preventing them from merging. This repulsion is proportional to the energy of the neighboring formant, making it more effective when formants are strong.
Additionally, the authors highlight the importance of the spectral analysis method. They advocate for the "true envelope algorithm" over standard linear prediction or cepstral smoothing for generating spectrograms, as it better approximates spectral peaks and is less sensitive to window placement relative to pitch periods.
Datasets
The paper describes experiments conducted on "18 repetitions of a small story ('La bise et le soleil') uttered by French speakers." This corpus includes 384 phonemes, with 39 being nasal vowels or nasalized sounds. No specific named dataset is mentioned, and no new benchmark is introduced.
Key Insights and Ideas
- Interdependency is Key: Treating formants as independent entities leads to control complexities and tracking errors, especially in challenging phonetic contexts. Making them influence each other directly simplifies the system and improves robustness.
- Repulsion is More Robust: The "repulsive tracks strategy" proved more effective than the "spectrogram partition strategy," particularly in cases where one formant track might be erroneously attracted to a strong spectral peak, potentially pulling it away from its true position. The repulsion helps maintain distinct tracks.
- Spectrogram Quality Matters: The choice of spectral analysis significantly impacts formant tracking. The "true envelope algorithm" is shown to be superior for this task because it more accurately represents spectral peaks, which are the targets for formant tracking.
- Simplified Initialization: By incorporating interdependency, the need for highly precise and pre-labeled initial formant tracks is reduced, as the dynamic interactions can correct initial errors more effectively.
Key Results
The paper demonstrates that the proposed concurrent curve strategies, particularly the repulsive tracks strategy, lead to more robust formant tracking. This is evidenced by:
- Improved Handling of Nasal Vowels: The repulsive strategy corrects weaknesses seen in the partition strategy, where nasalized sounds (like /yRã/) caused errors due to formants being too close.
- Qualitative Improvements: Figure 2 shows a comparison between the partition strategy (a) and the repulsion strategy (b), where (b) clearly shows more accurate and separated formant tracks for a challenging sequence.
- Reduced Control Complexity: The interdependency inherently manages interactions, reducing the need for complex, ad-hoc control logic.
Quantitative results are not explicitly detailed with metrics like accuracy percentages or error rates, but the qualitative examples and descriptions strongly suggest improved performance.
Limitations and Future Work
- Sensitivity to Nasalization: While improved, the system still struggles with "very marked" nasalization where extra spectral peaks appear below 3000 Hz, leading to erroneous results. The authors note that in such cases, tracks may jump between peaks due to amplitude differences.
- Formant Number Mismatch: The success of formant tracking is still dependent on the adequacy between the number of formants tracked and the actual number of spectral lines. Nasal sounds, in particular, can present extra spectral lines, leading to missed formants.
- Future Work: The authors plan to investigate using the ratio of energy integrated by formant tracks to evaluate results. This evaluation could help in signal segmentation and determining the optimal number of formant tracks needed.
Summary updated