Enhanced Speech Emotion Recognition Incorporating Speaker-Sensitive Interactions in Conversations

Jiachen Luo, Huy Phan, Lin Wang, Joshua D. ReissICME 2024 · OpenReview

Abstract

Accurately detecting emotions in conversation is a necessary yet challenging task due to the complexity of emotions and dynamics in dialogues. The emotional state of a speaker can be influenced by many different factors, such as interlocutor stimulus, dialogue scene, and topic. In this work, we propose a conversational speech emotion recognition method to deal with capturing attentive contextual dependency and speaker-sensitive interactions. First, we use a pretrained WavLM model to extract frame-based audio representation in individual utterances. Second, an attentive bi-directional gated recurrent unit (GRU) models contextual-sensitive information and explores listener dependency and speaker influence jointly in a simple, fast, parameter-efficient way. The experiments conducted on the standard conversational dataset MELD demonstrate the effectiveness of the proposed method when compared against state-of the-art methods.
BibTeX
@inproceedings{luo2024enhanced,
  author = {Jiachen Luo and Huy Phan and Lin Wang and Joshua D. Reiss},
  title = {{Enhanced Speech Emotion Recognition Incorporating Speaker-Sensitive Interactions in Conversations}},
  booktitle = {ICME},
  year = {2024},
  url = {https://openreview.net/forum?id=27F8q0VqK9},
}

Loading related content...

Figure