Abstract
Speech Emotion Recognition (SER) models often experience significant performance degradation when deployed on datasets that differ from their training source due to variations in recording environments, languages, and speaker characteristics. This paper introduces Transfer Subspace Learning (TRaSL), a novel framework designed for unsupervised cross-corpus SER. TRaSL maps features from disparate domains into a shared "label space" while simultaneously aligning their marginal distributions and preserving the underlying data structure. Experimental evaluations across four benchmark corpora—EmoDB, eNTERFACE, AFEW 4.0, and IEMOCAP—demonstrate that TRaSL consistently outperforms state-of-the-art domain adaptation techniques, achieving superior accuracy and stability in cross-corpus scenarios.
Introduction
The primary challenge in real-world Speech Emotion Recognition (SER) is the domain mismatch between training (source) and testing (target) data. While models achieve high accuracy on intra-corpus tasks, their performance often drops significantly when applied to a target domain with different acoustic properties, recording equipment, or languages. This phenomenon, known as "performance degeneration," is a major hurdle for deploying SER systems in diverse environments.
This paper addresses the specific challenge of unsupervised cross-corpus SER. In this scenario, researchers have access to a labeled source dataset but a target dataset that is entirely unlabeled. The goal is to bridge the gap between these different marginal probability distributions so that a classifier trained on the source can accurately predict emotions in the target.
The TRaSL Framework
The authors propose TRaSL (Transfer Subspace Learning), which focuses on learning a projection matrix to transform features into a common, discriminative subspace.
The framework integrates several sophisticated components to handle domain shift:
- Label Space Projection: Unlike standard methods that map to an arbitrary latent space, TRaSL maps features directly toward a "label space" defined by the source categories. By aligning the projected features with the label matrix, the model ensures that the resulting subspace is inherently discriminative for the classification task.
- Distribution Alignment via MMD: To bridge the domain gap, the model utilizes the Maximum Mean Discrepancy (MMD) criterion. MMD is a non-parametric metric that measures the distance between two distributions in a Reproducing Kernel Hilbert Space. By minimizing MMD, TRaSL reduces the distance between the mean feature vectors of the source and target domains.
- Structure Preservation: Beyond matching means, TRaSL minimizes the difference between the covariance matrices of the two domains. This ensures that the underlying geometric structure and "spread" of the data are preserved during the transfer, which is crucial for maintaining performance across different recording environments.
- Feature Selection with $L_{2,1}$-norm: The framework incorporates $L_{2,1}$-norm regularization on the projection matrix. This encourages row-sparsity, allowing the model to automatically identify and select the most important features for emotion recognition while discarding noisy or irrelevant ones.
- Optimization: The complex objective function is solved iteratively using the Inexact Augmented Lagrange Multiplier (IALM) method, ensuring efficient and stable convergence.
Experimental Evaluation
The researchers evaluated TRaSL using four widely recognized speech emotion corpora:
- EmoDB: A German database with 535 samples across 7 emotions.
- eNTERFACE: An audiovisual database (audio only used) with 1,287 samples across 6 basic emotions.
- AFEW 4.0: An "in-the-wild" dataset collected from movies, representing real-world challenges.
- IEMOCAP: A large-scale dataset from USC containing over 5,500 utterances across 4 primary emotions.
The team conducted 12 distinct cross-corpus experiments (e.g., training on IEMOCAP and testing on EmoDB) to rigorously test the model's adaptability.
Key Results
TRaSL was compared against a baseline SVM and five state-of-the-art domain adaptation methods: KMM, KLIEP, uLSIF, DALSR, and DoSL.
- Superior Accuracy: TRaSL achieved the highest overall results, with an average Weighted Average Recall (WAR) of 42.75% and an Unweighted Average Recall (UAR) of 41.08% across all experiments.
- Significant Gains: In the IEMOCAP to EmoDB transfer, TRaSL achieved a WAR of 65.49%, significantly outperforming the baseline SVM's 41.30%.
- Consistency: TRaSL achieved the best results in 8 out of the 12 experimental cases, proving to be a more stable and reliable framework for cross-corpus tasks compared to existing methods.
Discussion and Key Insights
A major innovation of TRaSL is the use of the "Label Space" as a bridge. By forcing both domains into a space representing emotional categories, the model naturally aligns target data with source decision boundaries. Furthermore, the results indicate that matching the "structure" (covariance) of the data is as crucial as matching the "means" for maintaining performance across different recording environments.
The authors also observed that TRaSL demonstrated robustness to class imbalance. In datasets like EmoDB, which are heavily biased toward certain emotions, standard models often get "tricked" into predicting the majority class. TRaSL’s subspace approach proved more resilient to these imbalances, maintaining better recall across all emotion categories.
Conclusion
TRaSL provides a robust solution for unsupervised cross-corpus SER by combining label space projection with distribution alignment and structure preservation. While highly effective, the current linear model has limitations when the source dataset is very small. Future work could explore non-linear projections via Deep Learning (CNNs or Transformers) to capture more complex relationships in audio data, as well as more sophisticated feature selection techniques to further refine the "label space" mapping.
Summary updated