MELD-ST: An Emotion-aware Speech Translation Dataset

Sirou Chen, Sakiko Yahata, Shuichiro Shimizu, Zhengdong Yang, Yihang Li, Chenhui Chu, Sadao Kurohashi

Abstract

Emotion plays a crucial role in human conversation. This paper underscores the significance of considering emotion in speech translation. We present the MELD-ST dataset for the emotion-aware speech translation task, comprising English-to-Japanese and English-to-German language pairs. Each language pair includes about 10,000 utterances annotated with emotion labels from the MELD dataset. Baseline experiments using the SeamlessM4T model on the dataset indicate that fine-tuning with emotion labels can enhance translation performance in some settings, highlighting the need for further research in emotion-aware speech translation systems.
BibTeX
@inproceedings{chen2024meld,
  author = {Sirou Chen and Sakiko Yahata and Shuichiro Shimizu and Zhengdong Yang and Yihang Li and Chenhui Chu and Sadao Kurohashi},
  title = {{MELD-ST: An Emotion-aware Speech Translation Dataset}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.601},
}
Figure