Language Independent Emotion Quantification using Non linear Modelling of Speech

Uddalok Sarkar, Sayan Nag, Chirayata Bhattacharya, Shankha Sanyal, Archi Banerjee, Ranjan Sengupta, Dipak GhosharXiv 2021 · arXiv

Abstract

Traditional speech processing relies on linear assumptions that often overlook the inherent complexity of human vocal production. This paper proposes a non-linear approach to emotion recognition using multifractal analysis. By applying Multi-Fractal Detrended Fluctuation Analysis (MFDFA) and Wavelet Transform Modulus Maxima (WTMM), we extract emotional "fingerprints" from speech signals. Our model, validated on English and German datasets, achieves a 96.3% average accuracy in distinguishing happy, sad, and neutral states, suggesting that non-linear features provide a robust, language-independent basis for emotion quantification.

Introduction

Standard emotion recognition systems typically utilize Mel-frequency cepstrum coefficients (MFCCs) or Linear Prediction Cepstrum Coefficients (LPCCs). These methods assume the human vocal tract is a linear, time-varying system. However, speech production is fundamentally non-linear and chaotic, involving complex air-puff oscillations and time-varying resonances. Linear models often discard high-frequency information and subtle complexities essential for emotional nuance. Furthermore, these systems frequently struggle with cross-linguistic generalization. This research seeks a mathematical "fingerprint" of emotion that remains consistent across different languages.

Methodology: Non-linear Multifractal Analysis

The researchers employed Non-linear Multifractal Analysis to quantify signal complexity. Two primary frameworks were used:

  1. Multi-Fractal Detrended Fluctuation Analysis (MFDFA): Detects long-range correlations in non-stationary time series to calculate a "singularity spectrum."
  2. Wavelet Transform Modulus Maxima (WTMM): Uses wavelets to identify local maxima across scales, effectively uncovering the multifractal distribution.

Statistical features, such as the area under the curve in the "low fluctuation region" and spectral width, were extracted from these spectra and fed into a Support Vector Machine (SVM) with a linear kernel for classification.

Multifractal spectra f(alpha) vs alpha showing the distinct clustering of different emotional states based on their singularity exponents.

Experimental Setup

The model's language independence was tested using two databases:

  • EMO-DB (German): Ten actors performing sentences in seven emotional states.
  • TESS (English): 2,800 voice samples from two actresses across various emotional contexts.

The study focused on three core states: Happiness, Sadness, and Neutral.

Results and Discussion

The analysis confirmed that speech is "strongly multifractal," with a spectral width consistently exceeding 0.5. A key finding was that emotions are most distinguishable in the "low fluctuation" region of the multifractal spectrum. In this region, happy speech typically occupies low Holder exponent values, sad speech occupies high values, and neutral speech falls in between.

The WTMM method generally outperformed MFDFA, providing clearer segregation of emotional attributes. The system achieved an average classification accuracy of 96.3%. In a specific test run of 36 clips (12 per emotion), the model reached 97.2% accuracy.

Confusion matrix for the three-class emotion recognition task, showing a high overall accuracy of 97.2%.

Conclusion

This study demonstrates that non-linear, multifractal modeling of speech provides a highly accurate and language-independent method for emotion recognition. While the pilot study focused on three primary emotions in controlled environments, future work will expand to include more nuanced emotional states (e.g., fear, boredom) and test the model's robustness in noisy, real-world settings and across tonal languages.

Summary updated

Abstract

At present emotion extraction from speech is a very important issue due to its diverse applications. Hence, it becomes absolutely necessary to obtain models that take into consideration the speaking styles of a person, vocal tract information, timbral qualities and other congenital information regarding his voice. Our speech production system is a nonlinear system like most other real world systems. Hence the need arises for modelling our speech information using nonlinear techniques. In this work we have modelled our articulation system using nonlinear multifractal analysis. The multifractal spectral width and scaling exponents reveals essentially the complexity associated with the speech signals taken. The multifractal spectrums are well distinguishable the in low fluctuation region in case of different emotions. The source characteristics have been quantified with the help of different non-linear models like Multi-Fractal Detrended Fluctuation Analysis, Wavelet Transform Modulus Maxima. The Results obtained from this study gives a very good result in emotion clustering.
BibTeX
@misc{sarkar2021language,
  author = {Uddalok Sarkar and Sayan Nag and Chirayata Bhattacharya and Shankha Sanyal and Archi Banerjee and Ranjan Sengupta and Dipak Ghosh},
  title = {{Language Independent Emotion Quantification using Non linear Modelling of Speech}},
  year = {2021},
  eprint = {2102.06003},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2102.06003},
}
Figure