MemEmo: Evaluating Emotion in Memory Systems

Peng Liu, Zhen Tao, Jihao Zhao, Ding Chen, Yansong Zhang, Cuiping Li, Zhiyu Li, Hong ChenarXiv 2026 · arXiv

TL;DR

Researchers have introduced MemEmo, a new benchmark designed to evaluate how effectively Large Language Model (LLM) memory systems process emotional information. Along with the HLME dataset, the study reveals a significant "affective gap": even state-of-the-art memory systems score below 45% on emotional intelligence tasks, struggling particularly with tracking emotional fluctuations and resolving conflicting affective states over long durations.

What Problem Does This Paper Address?

While LLMs have become incredibly capable, they suffer from "context loss" during long-term interactions. To solve this, developers use memory systems that store and retrieve past information. However, most current evaluations of these systems focus on factual retrieval (e.g., "What is the user's favorite color?").

In applications like AI companionship or personalized mental health support, emotional context is just as vital as factual data. If a user was upset about a job loss ten sessions ago, an empathetic AI should remember that emotional weight. This paper addresses the lack of a rigorous framework to measure affective intelligence—the ability to extract, update, reason about, and detect conflicts in emotional data within long-term memory.

Methods

The authors propose the MemEmo evaluation framework, which categorizes affective processing into four key dimensions:

  1. Emotional Information Extraction (EIE): Can the system identify the emotion type, polarity, and intensity from a dialogue?
  2. Emotion Memory Update (EMU): Can the system decide when to update its internal emotional state and maintain stability against "memory drift"?
  3. Emotional Question Answering (EQA): Can the system synthesize historical emotional trends to answer complex "why" questions?
  4. Emotion Conflict Detection (ECD): Can the system spot when a user’s current emotional expression contradicts a previously established preference or fact (e.g., expressing a love for peanuts despite a recorded allergy)?

To evaluate these, they used an LLM-as-a-Judge framework (primarily GPT-4o-mini) to score the outputs of five representative memory systems: MemOS, MemoBase, Mem0, Mirix, and Letta.

Datasets

The authors constructed the Human-Like Memory Emotion (HLME) dataset. It is built on a four-stage pipeline that generates personas, binds them to 49 distinct emotion categories (based on the EARL taxonomy), and produces multi-turn dialogues.

  • HLME-Medium: ~2,000 dialogue turns across 95 sessions (179K tokens).
  • HLME-Large: ~3,100 turns across 165 sessions (1.0M tokens), designed to test robustness against long-context noise.
  • Granularity: Each turn is annotated with fine-grained labels for emotion type, intensity (Low/Medium/High), and causal logic.

Key Insights and Ideas

  • The Affective Ceiling: The best-performing systems only reached an aggregate score of about 45%. This suggests that current memory architectures are "emotionally blind" compared to human capabilities.
  • The Reasoning-Extraction Trade-off: Systems like MemoBase excel at raw extraction and updating due to streamlined profiling, but they lack deep emotional reasoning. Conversely, MemOS handles complex logic and conflict detection better but struggles with raw extraction.
  • Conflict Detection is the "Final Boss": Across all models, scores for Emotion Conflict Detection (ECD) were drastically lower than other tasks. Detecting irony or a "change of heart" over 100+ sessions remains a massive hurdle for AI.
  • Noise Sensitivity: Performance significantly degrades on the "Large" version of the dataset. As the context grows, "distractor noise" makes it harder for systems to retrieve the specific emotional "anchor facts" needed for accurate reasoning.

Key Results

  • Overall Performance: Aggregate scores ranged from 27.9% (Mem0) to 43.3% (Mirix) on the Medium dataset.
  • Retrieval Depth: Expanding retrieval from Top-20 to Top-50 snippets improved evidence recall but often decreased overall QA accuracy because it introduced too much irrelevant noise.
  • Efficiency: MemOS was the fastest in terms of execution time, while Letta had the highest computational cost. MemoBase proved most efficient at ingesting the massive 1M-token Large dataset.

Limitations and Future Work

The authors acknowledge that their current work is limited to textual dialogue. Human emotion is inherently multimodal, involving tone of voice, facial expressions, and body language.

Future directions include: * Extending the MemEmo framework to multimodal affective analysis. * Improving memory synchronization to better handle "emotional fluctuations" over time. * Developing architectures that can capture "subtle and dispersed" emotional signals rather than just explicit emotional keywords.

Summary updated

Abstract

Memory systems address the challenge of context loss in Large Language Models during prolonged interactions. However, as an essential tool for warm companionship and personalized services, the effectiveness of memory systems in processing affective information remains underexamined. To bridge this gap, we propose an emotion-enhanced memory evaluation benchmark to assess the performance of mainstream and state-of-the-art memory systems in integrating and harnessing emotional information. First, we design a benchmark called \textbf{MemEmo} to assess the affective reasoning and tracking capabilities of memory systems across four dimensions: emotional information extraction, emotional memory updating, emotional memory question-answering, and emotional conflict detection. Second, we construct the \textbf{H}uman-\textbf{L}ike \textbf{M}emory \textbf{E}motion (\textbf{HLME}) dataset to provide an empirical foundation for scrutinizing memory systems. Finally, We evaluate five representative memory systems on the HLME dataset, reporting 11 performance metrics across four dimensions. Experimental results reveal that aggregating scores for all tested memory systems is up to 45\%, far below 100\% scale, exposing a profound disparity between memory system and affective intelligence.
BibTeX
@misc{liu2026mememo,
  author = {Peng Liu and Zhen Tao and Jihao Zhao and Ding Chen and Yansong Zhang and Cuiping Li and Zhiyu Li and Hong Chen},
  title = {{MemEmo: Evaluating Emotion in Memory Systems}},
  year = {2026},
  eprint = {2611.20733},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20733},
}
Figure