A Two-Stage Attention Based Modality Fusion Framework for Multi-Modal Speech Emotion Recognition
Lists
Only you see this
Abstract
Recently, automated recognition and analysis of human emotion has attracted increasing attention from multidisciplinary communities. However, it is challenging to utilize the emotional information simultaneously from multiple modalities. Previous studies have explored different fusion methods, but they mainly focused on either inter-modality interaction or intra-modality interaction. In this letter, we propose a novel two-stage fusion strategy named modality attention flow (MAF) to model the intra- and inter-modality interactions simultaneously in a unified end-to-end framework. Experimental results show that the proposed approach outperforms the widely used late fusion methods, and achieves even better performance when the number of stacked MAF blocks increases.
BibTeX
@article{hu2021two,
author = {Dongni Hu and Chengxin Chen and Pengyuan Zhang and Junfeng Li and Yonghong Yan and Qingwei Zhao},
title = {{A Two-Stage Attention Based Modality Fusion Framework for Multi-Modal Speech Emotion Recognition}},
journal = {IEICE Trans. Information and Systems},
year = {2021},
doi = {10.1587/transinf.2021edl8002},
}Loading related content...