Abstract
This study investigates the relative importance of various acoustic features in estimating continuous emotional primitives—Valence, Activation, and Dominance—from speech. Utilizing the VAM Corpus of spontaneous German speech, the researchers evaluated 46 acoustic features categorized into Duration, Energy, MFCC, and Pitch. Through the application of elementary estimators (Robust Regression, Support Vector Regression, and Locally Linear Reconstruction) and fusion techniques, a consistent hierarchy of feature importance was established: MFCC > Energy ≈ Pitch > Duration. Furthermore, the study demonstrates that locally weighted fusion significantly enhances estimation accuracy over individual models.
Introduction
Traditional speech emotion recognition (SER) often focuses on classifying utterances into discrete categories like "happy" or "sad." However, psychological models suggest that emotions are more accurately represented as coordinates in a multi-dimensional space defined by primitives such as Valence (positivity/negativity), Activation (calmness/excitement), and Dominance (weakness/strength).
While the importance of prosodic features like pitch is well-documented for categorical classification, their role in estimating continuous emotional primitives is less clear. This paper addresses this gap by performing a rigorous analysis of acoustic feature importance for 3D emotion space estimation, determining which "ingredients" of speech are most critical for accurate predictions.
Methodology
The research framework involves three main stages: data preparation, elementary estimator construction, and estimator fusion.
Four categories of acoustic features were extracted and analyzed:
- Duration (5 features): Length of voiced and unvoiced segments.
- Energy (6 features): Statistics related to signal intensity (mean, max, quantiles).
- MFCC (26 features): Mel-frequency cepstral coefficients representing the spectral envelope.
- Pitch (9 features): Fundamental frequency (f0) statistics.
Three "elementary" estimators were employed: Robust Regression (RR) (global linear), Support Vector Regression (SVR) (global nonlinear), and Locally Linear Reconstruction (LLR) (local linear). To improve performance, two fusion strategies were tested: Average Fusion (AF) and Locally Weighted Fusion (LWF), with LWF adaptively weighting models based on local performance. The study utilized the VAM Corpus, containing 947 spontaneous German utterances evaluated by multiple human listeners to establish ground truth coordinates.
Results and Discussion
The analysis revealed a consistent hierarchy of feature importance across all three emotional primitives. MFCCs emerged as the most critical features, suggesting that the spectral envelope contains significant emotional information beyond simple prosody. In contrast, duration features were found to be the least influential.
As shown in the results, SVR generally outperformed other individual models. However, fusion models, particularly LWF, provided the most accurate estimations by leveraging the complementary strengths of local and global models. The performance of LLR was notably sensitive to the number of features, performing poorly with limited feature sets but becoming competitive as more features were included. The consistency of the feature ranking (MFCC > Energy ≈ Pitch > Duration) across Valence, Activation, and Dominance underscores the robustness of these findings.
Conclusion and Future Work
This study provides a clear hierarchy of acoustic feature importance for estimating continuous emotion primitives, highlighting the dominance of spectral features (MFCCs) over prosodic ones (duration). The effectiveness of locally weighted fusion further suggests that combining local and global modeling approaches can significantly improve SER systems.
Future research should explore whether these findings generalize across different languages and cultures. Additionally, the authors suggest investigating more advanced features, such as Mel Filter Banks (MFB), which may offer even greater effectiveness in capturing emotional nuances in speech.
Summary updated