Does Memory Need Graphs? A Unified Framework and Empirical Analysis for Long-Term Dialog Memory

Sen Hu, Yuxiang Wei, Jiaxin Ran, Zhiyuan Yao, Xueran Han, Huacan Wang, Ronghao Chen, Lei ZouarXiv 2026 · DOI · OpenReview

TL;DR

This paper tackles the inconsistent performance of graph-based dialog memory systems by proposing a unified framework to analyze and compare various design choices. Their extensive experiments reveal that foundational system settings, rather than just graph structures, significantly impact performance, leading to the identification of robust baselines for future research.

What Problem Does This Paper Address?

The increasing adoption of graph structures in dialog memory systems, inspired by human memory's associative nature, has led to a proliferation of methods. However, empirical results on their effectiveness are contradictory, making it difficult to pinpoint which design choices truly matter. This inconsistency stems from a lack of a unified analytical perspective, varied datasets, evaluation settings, and underspecified implementation details across studies. Consequently, it's challenging to interpret prior work and for system builders to make informed decisions. This paper aims to bridge this gap by providing a systematic, experimental analysis of dialog memory architectures, offering clarity on design choices and establishing reliable baselines.

Methods

The authors introduce a unified framework that decomposes dialog memory systems into core components: Memory Extraction, Memory Indexing, Memory Retrieval, and Question Answering. This framework is designed to accommodate both graph-based and non-graph approaches. They then conduct controlled, stage-wise experiments on two benchmarks, LongMemEval and HaluMem, to systematically compare various design choices within this framework. These choices include:

  • Memory Representation (Keys & Values): Analyzing different forms of memory units (e.g., summaries, factual statements, entities, triples) and how they are represented.
  • Memory Organization & Maintenance: Investigating how keys are organized (e.g., flat index vs. graph) and how memory is updated (e.g., add, update, noop operations).
  • Indexing & Retrieval Strategies: Comparing flat index vs. graph index structures and different retrieval mechanisms (e.g., entity activation, graph expansion).

Datasets

The paper uses two established benchmarks for evaluating long-term dialog memory:

  • LongMemEval (Wu et al., 2024): Focuses on memory retrieval and reasoning over extremely long dialog histories, with subsets S and M.
  • HaluMem (Chen et al., 2025): Designed to evaluate memory extraction, updating, and consistency, with Medium and Long subsets. Experiments were conducted on the Medium subset due to computational costs.

Key Insights and Ideas

  • Foundational Settings Matter Most: The most surprising finding is that underlying implementation details and foundational system settings (like the choice of extraction model, embedding model, and how keys are represented) have a substantial impact on performance, often overshadowing the specific architectural innovations like graph structures.
  • Graphs Aren't Always Better: While graph-based methods can offer advantages, inappropriate graph construction or retrieval strategies can actually degrade results. This highlights the need for careful design and selection of graph components.
  • Update Operations are Crucial: The "Add" operation for memory maintenance is shown to be vital for keeping memory indices up-to-date and improving end-to-end QA performance.
  • Entity Descriptions Outperform Names: When using graph structures, using entity descriptions as keys provides richer semantic context and leads to better retrieval and QA compared to using just entity names.
  • Flat Indices Can Be Robust: For certain scenarios, particularly with less capable models or when dealing with temporal reasoning, flat indices can offer a more concise and effective representation, avoiding the "distracting noise" that can sometimes accompany graph expansion.

Key Results

  • Retrieval Performance: Graph-based methods generally outperform flat indices on LongMemEval, especially on the longer M subset. For instance, using GPT-40-mini for graph construction achieved a Recall@10 of 0.9711 on LongMemEval-S.
  • QA Accuracy: For the "Value=session" setting, graph methods show better QA accuracy, reaching 0.892 with GPT-40-mini. However, for the "Value=Key" setting, flat methods can perform better, achieving 0.752 on LongMemEval-S.
  • Efficiency: Graph-based retrieval is consistently slower than flat retrieval, but the latency remains acceptable for interactive systems. Memory extraction for graphs is more expensive but manageable.
  • Strong Baselines: The paper identifies several stable and reliable configurations that serve as strong baselines, outperforming contemporary methods. For example, a graph-based approach using entity descriptions as keys achieved QA accuracy of 0.892 on LongMemEval-S.

Limitations and Future Work

  • Framework Coverage: The proposed framework covers most common memory systems but may not encompass all highly customized or novel architectures.
  • Exploration Scope: The experiments did not exhaustively explore all possible design choices, such as alternative key types, heterogeneous graphs, or approaches requiring additional training.
  • Dataset Specificity: Conclusions are drawn from two benchmarks (LongMemEval and HaluMem), and findings might vary on other datasets or real-world applications with different characteristics.
  • Model Configurations: The study focused on a limited set of backbone models (LLaMA-3.1-8B, GPT-40-mini/40), and results might differ with other model combinations.

Future work could involve more fine-grained comparative experiments, exploring the interplay between update operations, and investigating the impact of different graph structures and retrieval strategies in more detail.

Summary updated

Abstract

Graph structures are increasingly used in dialog memory systems, but empirical findings on their effectiveness remain inconsistent, making it unclear which design choices truly matter. We present an experimental, system-oriented analysis of long-term dialog memory architectures. We introduce a unified framework that decomposes dialog memory systems into core components and supports both graph-based and non-graph approaches. Under this framework, we conduct controlled, stage-wise experiments on LongMemEval and HaluMem, comparing common design choices in memory representation, organization, maintenance, and retrieval. Our results show that many performance differences are driven by foundational system settings rather than specific architectural innovations. Based on these findings, we identify stable and reliable strong baselines for future dialog memory research. Code are available at https://github.com/AvatarMemory/UnifiedMem
BibTeX
@misc{hu2026does,
  author = {Sen Hu and Yuxiang Wei and Jiaxin Ran and Zhiyuan Yao and Xueran Han and Huacan Wang and Ronghao Chen and Lei Zou},
  title = {{Does Memory Need Graphs? A Unified Framework and Empirical Analysis for Long-Term Dialog Memory}},
  year = {2026},
  eprint = {2601.01280},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.01280},
}
Figure