Flexibly Utilize Memory for Long-Term Conversation via a Fragment-then-Compose Framework

Cai Ke, Yiming Du, Bin Liang, Yifan Xiang, Lin Gui, Zhongyang Li, Baojun Wang, Yue Yu, Hui Wang, Kam-Fai Wong, Ruifeng XuEMNLP 2025 · OpenReview

Abstract

Large language models (LLMs) have made significant breakthroughs in extracting useful information from conversation history to enhance the response in long-term conversations. Summarizing useful information from historical conversations has achieved remarkable performance, which, however, may introduce irrelevant or redundant information, making it difficult to flexibly choose and integrate key information from different sessions during memory retrieval. To address this issue, we propose a Fragment-then-Compose framework, a novel memory utilization approach for long-term open-domain conversation, called FraCom. To be specific, inspired by the concept of proposition representation from Cognitive Psychology, we first represent the conversation history as a series of predicates plus arguments for propositional representation to preserve key information useful for memory ("Fragment”). Then, we compose propositional graphs for the conversation history based on the connection between shared arguments ("Compose”). During retrieval, we retrieve relevant propositions from the graph based on arguments from the current query. This essentially allows for flexible and effective utilization of related information in long-term memory for better response generation towards a query. Experimental results on four long-term open-domain conversation datasets demonstrate the effectiveness of our FraCom in memory utilization and its ability to enhance response generation for LLMs.

Introduction

While Large Language Models (LLMs) excel at short-term context, maintaining coherence in long-term, multi-session conversations remains a challenge. Current state-of-the-art approaches typically rely on summarization, where previous dialogues are condensed into short paragraphs. However, these summaries often suffer from redundancy and noise, including "fluff" that distracts the model. Furthermore, session-by-session summarization creates fragmented memories, making it difficult for models to connect related facts mentioned across disparate sessions.

To address these limitations, we propose FraCom (Fragment-then-Compose), a framework that mimics human cognitive processes by storing memory as granular, interconnected propositional fragments rather than monolithic text summaries.

Methodology: The FraCom Framework

The FraCom framework operates in three distinct stages: Fragment, Compose, and Retrieval.

The Fragment-then-Compose (FraCom) framework overview, illustrating the transition from raw conversation history to structured propositional graphs and subsequent memory retrieval.

1. Fragment Module

Inspired by propositional representation in cognitive psychology, the system extracts the "gist" of every sentence as a series of predicates and arguments. For instance, the sentence "Lincoln, who was president during the war, freed the slaves" is decomposed into structured fragments like (president-of, United States, Lincoln, war) and (freed, Lincoln, slaves). This provides a "Goldilocks" unit of information—more structured than keywords but less noisy than full sentences.

2. Compose Module

Extracted fragments are merged into a Propositional Graph. Nodes representing shared arguments (e.g., "Lincoln") are linked across different sessions. This creates a unified knowledge web, enabling the model to perform cross-session reasoning by traversing connections between related entities.

3. Memory Retrieval and Response Generation

During a new interaction, the system performs Plausible Retrieval. It encodes the arguments of the current query (using Sentence-BERT) and calculates semantic similarity against the graph nodes. This allows the system to pull the most relevant propositions—even if phrased differently—to serve as precise context for the LLM's response.

Experimental Evaluation

We evaluated FraCom on four diverse long-term open-domain conversation datasets: Conversation Chronicles (CC), Multi-Session Chat (MSC), GC (focused on memory retrieval), and LongMemEval (LME) (extremely long-term interactive memory).

Performance Metrics

The framework was compared against strong baselines including MemoChat, MemoryBank, and COMEDY. We utilized the M1 Score to measure the model's ability to retrieve only necessary information.

Performance comparison (M1 %) across four datasets (CC, MSC, GC, LME) over multiple conversation sessions, showing FraCom's consistent effectiveness in memory utilization.

As shown in the results, FraCom consistently outperforms baselines in memory utilization across various datasets and sessions. Key findings include:

  • Superior Generation: FraCom achieved higher scores in BLEU, ROUGE, and BertScore.
  • Efficiency: The framework reduced storage overhead by 7% to 30% compared to summary-based methods.
  • Human Preference: In evaluations using GPT-4o and human annotators, FraCom was rated significantly higher in Engagingness, Humanness, and Memorability.

Discussion and Limitations

Ablation studies confirmed that the "Compose" step is vital for linking facts across sessions. However, the framework introduces higher computational latency, increasing time cost per session by roughly 27% to 49%. Additionally, the system's performance is sensitive to the initial extraction quality and currently lacks a mechanism to handle conflicting memories (e.g., a user moving locations between sessions).

Conclusion

FraCom offers a novel approach to long-term conversation memory by leveraging structured propositional graphs. By moving away from noisy text summaries toward granular, interconnected fragments, it enables LLMs to achieve more surgical and contextually relevant memory retrieval, ultimately leading to more human-like and engaging long-term interactions.

Summary updated

Abstract

Large language models (LLMs) have made significant breakthroughs in extracting useful information from conversation history to enhance the response in long-term conversations. Summarizing useful information from historical conversations has achieved remarkable performance, which, however, may introduce irrelevant or redundant information, making it difficult to flexibly choose and integrate key information from different sessions during memory retrieval. To address this issue, we propose a Fragment-then-Compose framework, a novel memory utilization approach for long-term open-domain conversation, called *FraCom*. To be specific, inspired by the concept of proposition representation from Cognitive Psychology, we first represent the conversation history as a series of predicates plus arguments for propositional representation to preserve key information useful for memory ("**Fragment**”). Then, we compose propositional graphs for the conversation history based on the connection between shared arguments ("**Compose**”). During retrieval, we retrieve relevant propositions from the graph based on arguments from the current query. This essentially allows for flexible and effective utilization of related information in long-term memory for better response generation towards a query. Experimental results on four long-term open-domain conversation datasets demonstrate the effectiveness of our *FraCom* in memory utilization and its ability to enhance response generation for LLMs.
BibTeX
@inproceedings{ke2025flexibly,
  author = {Cai Ke and Yiming Du and Bin Liang and Yifan Xiang and Lin Gui and Zhongyang Li and Baojun Wang and Yue Yu and Hui Wang and Kam-Fai Wong and Ruifeng Xu},
  title = {{Flexibly Utilize Memory for Long-Term Conversation via a Fragment-then-Compose Framework}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=68ZFPSbbtP},
}
Figure