TL;DR
This paper introduces a novel approach to automatically classify phonation modes (like breathy, modal, flow, and pressed) in singing voices. By leveraging newly derived excitation source features and Single Frequency Filtering Cepstral Coefficients (SFFCCs), the authors achieve higher classification accuracy than traditional methods, overcoming limitations associated with inverse filtering in complex singing signals.
What Problem Does This Paper Address?
Singing voices are rich with expressive nuances, and a key component of this expressiveness is the "phonation mode" – the way the vocal folds vibrate and interact with the vocal tract. These modes, often categorized as breathy, modal (normal), flow, and pressed, convey different emotions and vocal qualities. Accurately detecting and classifying these modes is crucial for applications like vocal disorder diagnosis, music information retrieval, and even aiding singing instruction.
Previous attempts to classify phonation modes often relied on voice quality features derived from inverse filtering. However, inverse filtering, a technique used to separate the vocal source from the vocal tract filter, struggles with the complex source-filter coupling present in singing, especially with high-pitched notes. This difficulty leads to inaccuracies in feature extraction and, consequently, in phonation mode classification. This paper aims to address this by proposing features that bypass the need for inverse filtering and are directly extracted from the audio signal.
Methods
The paper proposes two main sets of features:
- Excitation Source Features (from Modified Zero Frequency Filtering - ZFF): Instead of relying on inverse filtering, the authors extract features directly from the excitation source. They use a modified ZFF method, which is known to provide robust estimates of glottal closure instants (GCIs) even in the presence of rapid pitch variations common in singing. From these GCIs, they derive features like:
- Strength of Excitation (SoE): Proportional to the rate of vocal fold closure.
- Energy of Excitation (EoE): Reflects vocal effort.
- Loudness: Captures the abruptness of glottal closure.
- ZFF Signal Energy: Energy of the ZFF signal around GCIs.
- Single Frequency Filtering Cepstral Coefficients (SFFCCs): These features are derived using the Single Frequency Filtering (SFF) method. The SFF method aims to extract the amplitude envelope of the signal with adjustable spectro-temporal resolution. The process involves:
- Pre-emphasizing the input signal.
- Shifting the signal's frequency content using a complex exponential.
- Passing the frequency-shifted signal through a single-pole filter.
- Estimating the amplitude envelope of the filtered signal. Finally, the cepstrum is computed from the log of this amplitude envelope, and the first 13 coefficients are retained as SFFCCs. These are computed at GCI locations for efficiency.
The classification itself is performed using Support Vector Machines (SVMs) with a Radial Basis Function (RBF) kernel, employing a 10-fold cross-validation strategy.
Datasets
The study utilizes a phonation dataset described in [1]. This dataset consists of sustained vowels sung by a professional soprano singer in Russian. It includes recordings of nine different vowels (A, AE, I, O, U, UE, Y, OE, E) with pitches ranging from A3 to G5. The dataset contains 763 recordings, with an average duration of 1.3 seconds. The phonation modes present are breathy, neutral, flow, and pressed, based on Sundberg's definitions.
Key Insights and Ideas
- Bypassing Inverse Filtering is Key: The core insight is that direct extraction of excitation source features and the use of SFFCCs can overcome the limitations of inverse filtering in singing voice analysis, leading to more accurate phonation mode classification.
- Complementary Information: Excitation source features and SFFCCs provide complementary information. Combining them significantly improves classification accuracy compared to using either set alone.
- VQ Features Struggle with Singing: Traditional voice quality (VQ) features, often derived from inverse filtering, show poor discrimination for singing voice, likely due to the challenges of accurate inverse filtering in high-pitched singing.
- SFFCCs Outperform VQ and MFCCs Individually: SFFCCs alone achieve better classification accuracy than VQ features and even Mel-Frequency Cepstral Coefficients (MFCCs) individually.
- Combined Approach is Optimal: The best performance is achieved by combining the proposed excitation features, SFFCCs, and MFCCs, suggesting a comprehensive feature set is most effective.
Key Results
The paper reports classification accuracies using 10-fold cross-validation with SVMs:
- VQ Features: Mean accuracy of 29.18%.
- MFCCs: Mean accuracy of 61.05%.
- Excitation Features: Mean accuracy of 52.11%.
- SFFCCs: Mean accuracy of 65.24%.
- Excitation + SFFCCs: Mean accuracy of 67.12%.
- MFCC + Excitation + SFFCCs: Achieved the highest mean accuracy of 70.92%.
Confusion matrices reveal that while the combined approach significantly reduces confusion between breathy and modal voices, confusion between flow and tense voices persists, suggesting areas for further improvement.
Limitations and Future Work
- Limitations:
- The study acknowledges that while the combined feature set (MFCC + Excitation + SFFCC) yields the highest accuracy, confusion still exists between flow and tense phonations, and also between breathy and modal voices.
- The reliance on a single singer's recordings might limit the generalizability of the findings to a wider range of singers and vocal styles.
- The paper notes that existing VQ features struggle with singing, implying that improvements in inverse filtering for singing or alternative VQ feature extraction methods could be beneficial.
- Future Work:
- The authors suggest exploring features that can better capture the effect of excitation on the vocal tract system, which could further improve the discrimination of challenging phonation pairs (flow/tense, breathy/modal).
- Investigating features that can capture the interaction between the vocal tract and the glottal source, especially for discriminating between breathy and modal, and flow and tense voices.
- The proposed features, which do not rely on the source-filter model, are suggested to be useful for analyzing continuous speech and singing. Further research could explore their application in these broader contexts.
Summary updated