CoMPM: Context Modeling with Speaker's Pre-trained Memory Tracking for Emotion Recognition in Conversation

Joosung Lee, Wooin LeeNAACL 2022 · DOI

Introduction

Emotion Recognition in Conversation (ERC) is a complex task that extends beyond simple sentiment analysis. It requires a system to interpret nuances like sarcasm, frustration, or anger by considering both the dialogue history and the specific persona of the speakers involved. Unlike standalone sentence analysis, ERC must account for how a speaker's emotional state evolves in response to others.

An example of emotional dynamics in conversation, where understanding the speaker's state and dialogue history is crucial for accurate recognition.

Historically, high-performing ERC models have relied on external "commonsense" knowledge bases such as ConceptNet or ATOMIC. While effective, these resources are primarily available in English, creating a significant barrier for multi-lingual applications. CoMPM addresses this by leveraging the "compressed knowledge" within pre-trained language models (PLMs) to track speaker states directly, eliminating the need for external structured data.

Methodology: The CoMPM Framework

The CoMPM architecture is built around two synergistic modules designed to capture different aspects of conversational dynamics:

  1. Context Embedding Module (CoM): Acting as a "broad listener," this module utilizes a Transformer encoder (such as RoBERTa) to process the entire dialogue history. It identifies the general thematic and emotional flow of the conversation.
  2. Pre-trained Memory Module (PM): This "speaker specialist" module focuses exclusively on the current speaker's previous utterances. By passing these through a PLM and a Gated Recurrent Unit (GRU), the model maintains a running "memory" of the speaker's linguistic patterns and emotional baseline.
The CoMPM architecture, featuring the Context Embedding Module (CoM) and the Pre-trained Memory Module (PM) for speaker tracking.

The outputs of these modules are integrated using a projection matrix ($W_p$), allowing for flexibility in choosing different underlying architectures for the CoM and PM components (e.g., combining RoBERTa with BERT).

Experimental Results

The researchers evaluated CoMPM across four major English benchmarks: IEMOCAP, DailyDialog, MELD, and EmoryNLP. The model achieved state-of-the-art (SOTA) or near-SOTA performance on all datasets, notably outperforming several models that utilize external knowledge bases.

  • IEMOCAP: Achieved a Weighted-Avg F1 of 69.46.
  • MELD: Reached an F1 score of 66.52.

A critical finding was the model's performance on a private Korean dataset. By simply swapping the underlying PLM for a Korean-specific one, CoMPM showed a significant performance boost (from 57.46 to 60.66 F1) upon adding the memory module, demonstrating its robust cross-lingual capabilities without needing specialized Korean knowledge graphs.

Key Insights

  • PLMs as Knowledge Sources: Large pre-trained models contain sufficient internal knowledge to substitute for structured databases in emotional context tracking.
  • Efficiency of Frozen Modules: In scenarios with limited training data, "freezing" the memory module (CoMPM(f)) often yielded superior results, suggesting that the pre-trained features are highly transferable and act as a stable knowledge extractor.
  • The Primacy of Context: While speaker memory provides a vital refinement, ablation studies confirm that the Context Module (CoM) remains the foundational driver of accuracy. Memory helps refine the prediction, but the dialogue flow provides the necessary foundation.

Conclusion

CoMPM represents a significant step toward language-agnostic emotional AI. By replacing external knowledge graphs with internal speaker memory tracking, the framework achieves high accuracy while remaining adaptable to any language with a high-quality pre-trained model. Future work will likely explore more sophisticated dialogue context modeling to further enhance these results.

Summary updated

BibTeX
@inproceedings{lee2022compm,
  author = {Joosung Lee and Wooin Lee},
  title = {{CoMPM: Context Modeling with Speaker's Pre-trained Memory Tracking for Emotion Recognition in Conversation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.naacl-main.416},
}
Figure