TL;DR
HabitusMem introduces a "belief-state" approach to LLM memory, moving beyond simple retrieval of past interactions to maintaining a structured, evolving model of user preferences. By representing memory as a set of revisable beliefs—each with explicit evidence, scope, and status—it enables agents to navigate contradictory or outdated history and act on what the user wants now.
What Problem Does This Paper Address?
Most current LLM agents treat memory as a storage problem: they use Retrieval-Augmented Generation (RAG) to pull relevant snippets from past chats or maintain summaries of "what happened." However, these systems often fail when user preferences evolve.
If a user says "I hate spicy food" in January but later asks for "extra chili" in a specific Thai restaurant review in February, a standard RAG system might retrieve both, leaving the agent confused. Current systems lack a way to distinguish between a standing preference, a one-off exception, and a preference that has been superseded or retracted. As shown in Figure 1, content-based memory passes on fragments of history, but leaves the difficult task of inference—deciding which preference currently holds—to the agent.
Methods
The authors propose HabitusMem, a trained "writer" (implemented as a LoRA adapter for Qwen3.5-9B) that sits between the raw history and a frozen agent. Instead of raw text, it generates structured belief records in the format:
claim | evidence | scope | status
- Claim: The inferred preference or fact (e.g., "Prefers mid-priced reds").
- Evidence: Citations of specific past messages that support the claim.
- Scope: The context where the preference applies (e.g., "only for work trips").
- Status: Whether the belief is active, superseded by a newer one, or retracted by the user.
The system is trained in two stages. First, a "parent writer" learns the record syntax. Second, the model is trained on four auxiliary tasks:
- Belief Inference: Writing beliefs from raw history.
- State Update: Revising existing beliefs as new evidence arrives.
- Request Projection: Selecting only the beliefs relevant to the current user query.
- Action Projection: Selecting beliefs required for specific tool-use constraints.
Datasets
The authors evaluate HabitusMem on two primary benchmarks:
- PersonaMem-v2: A dataset for personalized question answering involving long interaction histories, changing preferences, and explicit requests to "forget" information.
- VitaBench 2.0: A benchmark for personalized tool use (e.g., booking travel or ordering food) where preferences are often implicit in past behavior.
Key Insights and Ideas
- Habitus as Latent State: Borrowing from Pierre Bourdieu’s sociology, the authors model the user as a "habitus"—a set of durable but revisable dispositions. This justifies treating memory not as a database of facts, but as a "belief state" that is updated by every new interaction.
- Request-Conditioned Memory: One of the biggest performance gains comes from projection. By filtering the memory to only include beliefs relevant to the current request, the system reduces noise and prevents irrelevant past data from "distracting" the agent.
- Explicit Retraction vs. Deletion: Unlike systems that simply delete old data, HabitusMem keeps retracted preferences but marks their status. This allows the agent to recognize when a user is specifically asking not to do something they used to like, which is a more robust way to handle "unlearning."
Key Results
HabitusMem consistently outperformed established memory systems like RAG, Mem0, and A-MEM across different agent families (Qwen and DeepSeek).
- Personalized Answering: On PersonaMem-v2, HabitusMem achieved an overall accuracy of 0.537 (with Qwen), roughly 4.8 points higher than the strongest baseline. Notably, it excelled in "Knowledge" scenarios where raw history often interferes with the agent's internal knowledge.
- Tool Use Consistency: On VitaBench 2.0, HabitusMem achieved the highest "all-success" rate (Pass⁴), meaning it was the most consistent at completing complex tasks across multiple trials. It even outperformed giving the agent the "Full Context" of the history, proving that a curated belief state is more effective than raw data.
Limitations and Future Work
- Fixed Input Budget: The system uses a position-based rule to select which parts of the raw history the "writer" sees. Very long histories might still result in lost evidence if it falls outside this window.
- Imperfect Extraction: While better than baselines, the belief extraction process isn't perfect, especially for highly nuanced or deeply implicit preferences.
- Task Breadth: The current evaluation focuses on QA and tool use. The authors suggest future work should explore broader applications like personalized email composition and complex scheduling.
Summary updated