TL;DR
Weaver is a new framework for personalized Retrieval-Augmented Generation (RAG) that treats user history like a detective story rather than a simple database. By using a dual-layer graph to track "information gaps" and follow textual clues across conversations, it "weaves" together scattered evidence to understand not just what a user likes, but the specific context and time in which those preferences apply.
What Problem Does This Paper Address?
Standard RAG systems often struggle with personalization because user preferences are rarely stated in a single, neat sentence. Instead, they are fragmented across months of chat history and context-dependent.
For example, a user might say "I'm going on a solo trip" in one conversation and "I prefer quiet hotels when I'm alone" in another. A basic retriever might find the first part but miss the second, leading to a generic recommendation. Furthermore, preferences change: a user who loved "lively" hotels last year might have updated their preference to "quiet" ones this year. Standard RAG often conflates these conflicting signals or fails to realize that a retrieved fragment is only a partial clue that requires further searching.
Methods
Weaver introduces a Dual-Layer Retrieval and Reasoning Graph to manage this complexity:
- Upper Layer (Intent Dependency Graph): This layer acts as the "planner." It uses an LLM to decompose a user's request into sub-questions (e.g., "What is the travel party?" and "What are the hotel preferences for this party?"). It tracks Dependency Information Gaps—essentially keeping a checklist of what it still needs to know to provide a grounded answer.
- Lower Layer (Memory-Conditioned Retrieval Trajectory Graph): This is the "detective" layer. Instead of a one-shot search, it performs Incremental Retrieval. If a retrieved fragment contains a clue (e.g., a mention of a "Hilton stay"), Weaver uses that clue to formulate a new, targeted query to find the missing pieces of that specific memory.
Connecting these layers is Scope-Aware Compositional Reasoning. This module jointly interprets accumulated fragments, checking their temporal and situational "scope." It determines if fragments complement each other, provide alternative options, or if a newer memory supersedes an older one.
Datasets
The authors evaluated Weaver on several challenging benchmarks:
- PersonaMem: A dataset featuring 32K-token user histories where preferences evolve over time.
- PrefEval: A benchmark for following preferences in conversational contexts.
- MuSiQue & WebWalkerQA: Non-personalized multi-hop reasoning tasks used to demonstrate that Weaver’s "clue-following" logic works for general factual retrieval as well.
Key Insights and Ideas
- Relevance $\neq$ Applicability: Just because a memory is semantically related to a query doesn't mean it applies to the current situation. Weaver explicitly reasons about situational scope (e.g., "solo" vs. "with friends").
- Retrieval as a Trajectory: Effective retrieval isn't a single jump; it's a path. A retrieved document is often just a stepping stone to the actual evidence needed.
- Closing the Loop: By returning "unresolved gaps" from the reasoning module back to the retrieval module, the system avoids "hallucinating" an answer when evidence is still missing.
Key Results
Weaver significantly outperformed existing state-of-the-art memory-augmented models:
- Accuracy Gains: On PersonaMem, Weaver achieved 86.42% accuracy, beating the strongest baseline (MemCoE) by 16.01 percentage points. On PrefEval, it reached 91.10%, a 7.56-point improvement.
- Efficiency: Despite its iterative nature, Weaver is surprisingly efficient. It uses only ~56% of the tokens required by MemCoE while delivering better results and maintaining a similar end-to-end latency (~327s vs 360s).
- Robustness: Diagnostic tests showed that Weaver’s advantage grows as the number of required memories increases, proving its strength in "weaving" complex, multi-part evidence.
Limitations and Future Work
- Latency: While efficient in token usage, an end-to-end latency of over 5 minutes per request is still too slow for real-time consumer chatbots.
- LLM Dependency: The framework relies heavily on the reasoning capabilities of the underlying LLM (Qwen-27B in this study) to build and update the graphs.
- Future Directions: The authors suggest exploring ways to further distinguish between temporal updates (changing preferences) and situational variations (different preferences for different contexts) in their diagnostic protocols.
Summary updated