TL;DR
CALM (Conflict-Aware Lifecycle Memory) is a new framework that allows AI agents to manage long-term memories by separating what is stored from what is used for a specific question. By tracking the "lifecycle" of information—marking facts as active, superseded, or stale—it enables agents to accurately answer questions about the present without losing the ability to reconstruct the past.
What Problem Does This Paper Address?
Most current Large Language Model (LLM) agents use a "Retrieval-Augmented Generation" (RAG) approach: when you ask a question, the agent searches its history for semantically relevant snippets. However, semantic relevance does not equal query eligibility.
Consider a user who tells an agent, "I live in Seattle," and months later says, "I moved to Boston." If the user asks, "Where do I live now?", both statements are semantically relevant. A standard retriever might pull both, leading the LLM to provide an ambiguous or outdated answer. Conversely, if the system simply deletes the Seattle record to avoid conflict, it loses the ability to answer a historical question like, "Where did I live before moving to Boston?"
The authors identify three distinct properties of memory that existing systems conflate:
- Semantic Relevance: Is this related to the topic?
- Query-Independent Validity: Is this fact still true/reliable based on all evidence?
- Query-Conditioned Eligibility: Is this specific piece of info appropriate for this specific question (e.g., a current-state vs. a historical query)?
Methods
CALM tackles this through a three-stage architecture that decouples persistent memory preservation from query-time admission.
- Stage I: Evidence-Grounded Memory Adaptation: Raw interaction logs (dialogues, facts) are converted into canonical "memory cells." These cells include metadata like semantic type, temporal anchors, and provenance (where the info came from).
- Stage II: Lifecycle-Aware Memory Governance: This is the "brain" of the system. It builds a lifecycle graph where memories are linked by support, conflict, or dependency. Facts move through states: Candidate $\rightarrow$ Active $\rightarrow$ Stable. If a new fact contradicts an old one, the old one is marked Superseded. If a prerequisite fact is invalidated, the dependent fact becomes Stale.
- Stage III: Query-Conditioned Memory Gating: At query time, CALM uses a "gate" to decide which retrieved memories actually enter the LLM's context window. It evaluates a four-dimensional feature vector for each memory:
- Relevance: Topic match.
- Validity: The stored lifecycle state from Stage II.
- Compatibility: Does the memory's time/scope match the query's intent?
- Risk: Potential for conflict or invalid dependency.
The authors tested both a training-free version (CALM-ZS) and a version with a lightweight learned MLP gate (CALM).
Datasets
The framework was evaluated on two major benchmarks for long-term memory:
- LongMemEval-S: 500 questions across categories like user preferences, knowledge updates, and temporal reasoning.
- LoCoMo: A dataset of very long conversations (up to 100k tokens) with 1,540 non-adversarial questions covering single-hop, multi-hop, and temporal reasoning.
Key Insights and Ideas
- Superseded $\neq$ Deleted: The most important "aha" moment is that outdated information is still valuable. By keeping superseded records but "blocking" them from current-state queries, CALM achieves the best of both worlds: accurate present-day reasoning and perfect historical recall.
- Decoupling is Key: Moving the logic of "what to use" away from the retriever and into a dedicated governance/gating layer allows for much more granular control without needing to retrain the underlying LLM or retriever.
- Conflict as a Signal: Instead of seeing conflicts as errors to be resolved immediately, CALM treats them as structural signals that define the relationship between different pieces of evidence over time.
Key Results
CALM consistently outperformed state-of-the-art baselines like Mem0, MemOS, and EverMemOS across different LLM backbones (Qwen and GPT-4).
- Accuracy: On LongMemEval-S, CALM reached 79.60% accuracy, significantly higher than Mem0 (40.20%) and EverMemOS (66.40%).
- Historical Reconstruction: CALM showed a massive advantage here. While "Latest-Only" strategies fail historical queries and "Keep-All" strategies struggle with current-state conflicts, CALM excelled at both.
- Efficiency: CALM is particularly effective under tight context constraints. As shown in Figure 4, its performance lead is most pronounced when the memory-token budget is limited, proving it selects the right evidence rather than just more evidence.
Limitations and Future Work
The authors acknowledge several areas for improvement:
- Deterministic Rules: The current lifecycle governance relies on hand-crafted rules for conflict detection, which might miss subtle or implicit contradictions.
- Retrieval Dependency: If the initial retriever fails to find the relevant (even if superseded) memory, the gating stage can't fix it.
- Scalability: Maintaining a complex lifecycle graph for millions of memories in a lifelong learning scenario remains a challenge for future investigation.
- Multimodal: The current work is text-only; extending these lifecycle concepts to images or video is a promising next step.
Summary updated