Conversational assistants that look things up as they talk, a setup called retrieval-augmented generation (RAG), get worse as conversations grow longer. Benchmarks built for this setting all show the same drop after the fourth to sixth turn. One reason is that a single bad retrieved passage at turn three never leaves. Whether it sits in the raw history, gets folded into a running summary, or gets averaged into an entity's stored representation, the wrong fact keeps resurfacing in every later answer. We propose to store what the conversation has established about each entity (each person, product, place, or concept under discussion) in an explicit table of attributes. Each attribute carries its source and a confidence level. New retrieved claims are allowed to change an attribute only when they agree with, or are corroborated against, what the table already holds. The same table then plays three roles. It forms the retrieval query, it scores incoming passages for consistency, and it replaces the older dialogue history in the prompt.
The claim is not that an entity table helps; several systems already show some form of that. The claim is that the write rule is what makes multi-turn RAG robust: when a stored belief may be overwritten, and on what evidence. Existing dynamic entity memories update by unconditional averaging. Under that rule, a passage that is retrieved wrongly or mentions a confusable entity contaminates the memory permanently. A memory whose writes are gated should stop that contamination from carrying forward. It should also let the model run on a short history window, which avoids the self-repetition seen with long histories, without forgetting what turn one established.
Core Claim
The proposal sits where four research directions meet, and each plays a distinct role. Dynamic entity memory supplies the mechanism. External knowledge retrieval is where the contamination comes from. Contextual noise sensitivity defines the failure we measure. Multi-turn dialogue is the regime where that failure compounds.
The classic neural entity memories update by convex combination. Dynamic Entity Representations in Neural Language Models mixes each entity's previous embedding with the current hidden state, and Incremental Neural Coreference Resolution in Constant Memory uses learned weighted averages. DynamicER carries the same idea into RAG: it drifts each entity's representation toward newly linked mentions. These papers supply the slot idea. They also constrain our design, because their own limitation is ours to fix: a misrouted mention pollutes the target slot irreversibly.
GAM-RAG is the closest method donor. It applies a Kalman-inspired gain to sentence-level memory vectors inside a retrieval index. The gain lets uncertain memories change quickly and damps updates once they stabilize, and it survives 20% noise injected into its feedback signal. GAM-RAG, however, adapts a static index across a sequence of questions. It does not track discourse referents or their attributes within a single dialogue. We move its damping idea to attribute-level entity slots that live for one conversation. We also drive the gain by agreement between independent sources, rather than by an LLM-judge feedback signal.
Research Program
The memory holds one record per entity. Each record has a stable identity key, an approach taken from the key–value split in What do Entity-Centric Models Learn? and NE-Table. The record also holds a set of attribute entries. Every entry stores a value, a confidence $c \in [0,1]$, and a provenance tag: stated by the user, stated by the assistant, or retrieved from a passage with its ID.
At each turn, a small LLM step does three things. It links mentions in the user turn to records; for nicknames and low-overlap descriptions it falls back to DynamicER-style dense linking. It proposes candidate attribute writes from the retrieved passages. It then classifies each proposal against the stored entry as agreeing, new, or conflicting.
The write rule is the object of study. Agreeing writes raise confidence. New attributes enter at a confidence that depends on their source. A conflicting write does not overwrite the stored value. Instead it goes into a pending slot and replaces the stored value only after a second, independent source corroborates it. User-stated values are an exception and win immediately, because users can correct the record. Written as a gain, the update to a stored value $v$ from a proposed value $u$ is
$$v \leftarrow v + K\,(u - v), \qquad K = \frac{\sigma_v}{\sigma_v + \sigma_u},$$
Here $\sigma_v$ is the uncertainty of the stored belief, which shrinks as corroboration accumulates. $\sigma_u$ is the uncertainty of the incoming source: high for a single retrieved passage whose entity-consistency score is low, and low for an explicit user statement. For discrete attribute values we apply the gain as a threshold on accumulated support rather than as interpolation.
The memory is then read in three ways. First, the retrieval query is the current user turn plus the linked records. This replaces free-form query rewriting. Second, retrieved passages are reranked by how consistent they are with high-confidence attributes. A passage about a confusable entity, such as a sibling car model or a different person with the same nickname, is down-weighted; this applies at the dialogue level the isolation by entity that MES-RAG does inside a vector store. Third, the reader's prompt holds the last one or two turns plus the serialized relevant records. History beyond that window is dropped. This is the context clipping of Mitigating Conversational Inertia in Multi-Turn Agents, made safe because the facts established in turn one survive in the records.
We will implement this as a prompted pipeline around Llama-3.1-8B-Instruct, so that it can be compared directly with Ext2Gen, which reports results on that model. A second variant fine-tunes only the module that proposes and classifies writes.
Baselines
The naive reference point is full-history concatenation: every prior turn plus the top-$k$ retrieved passages, with the same retriever and reader. A second naive point is last-turn-only context. CORAL shows its "Last Response" variant underperforming. Together the two bound the trade-off between keeping information and repeating oneself.
The strongest history-management baseline is LLM summarization, as benchmarked in CORAL. It reaches the best citation precision there, 31.1%, and is the most natural competitor to a structured memory. We will run it on the same 8B reader, with the same token budget per turn as our serialized records.
The strongest multi-turn memory system in the file is CMT-RAG. It keeps a recurrent state for local discourse continuity and a directed graph of sub-question traces with paragraph IDs. It reports 41.73 EM on the MuMu-QA long-dialogue split, 6.07 points above direct RAG. It is the right competitor because it also persists structured cross-turn evidence; its unit is the sub-question trace rather than the entity attribute, and it has no defense against noise. We will report its published number. Where we can reproduce it, we will also rerun it under our noise injection with the same retriever.
The strongest single-turn denoiser is Ext2Gen. It is trained with DPO, a preference-optimization method, to extract evidence sentences before answering. On Llama-3.1-8B-Instruct it reports an average of 0.557 versus 0.473 for the default model, and it beats the standalone compressors Recomp, CompAct, and EXIT. We will run it per turn with CQR queries. Ext2Gen is also composable with our method, so we test the combination as well. Beating Ext2Gen+CQR means higher accuracy on turns five and later, and lower contamination persistence, at matched reader compute.
The ablations isolate the write rule. The table, linking, and read paths stay identical, and only the update changes. The variants are overwrite-on-write, convex averaging in the style of ENTITYNLM and DynamicER, a GAM-style continuous gain without the pending slot, and our gated rule. If gated writes do not beat convex averaging on contamination persistence, the core claim fails.
Why not just…?
Why not just rewrite each turn into a standalone query and add Ext2Gen-style extraction? This is the most serious threat, because it handles both retrieval and noise using components that already exist. Its weakness is persistence. Rewriting and extraction are both stateless per turn. Once a wrong answer appears at turn $k$, it sits in the history that the next rewrite conditions on, so the error enters the next query. MTRAG reports that rewriting still leaves later turns markedly weaker. RECOR attributes 63% of failures to later turns with implicit links between turns. Our contamination-persistence experiment tests this directly. If CQR+Ext2Gen recovers from an injected error as quickly as gated memory does, the memory adds nothing, and we will report that.
Why not just keep a running LLM summary? CORAL shows that summarization is the best of the simple strategies. CMT-RAG argues that summaries blur the specific evidence that retrieval needs. The inertia paper observes that summaries over-claim and conclude prematurely. A summary that absorbs a bad passage has no provenance and no way to un-learn it. We compare the two at equal token budgets and under injected noise.
Why not just ask the LLM to maintain a free-form entity table in its prompt? This is the honest null hypothesis for "structure helps." It is exactly our overwrite ablation: the LLM rewrites the table every turn. If it matches the gated rule, the contribution reduces to "use a table." We would then reframe the work as an evaluation study rather than a method.
Why not just apply GAM-RAG directly? GAM-RAG adapts sentence weights in a shared index across queries. It never forms a belief about "this user's laptop model" that later passages can contradict. Its continuous-gain variant is one of our ablations.
Proposed Experiments
The evaluation spans four conversational benchmarks. We use the MTRAG human conversations, which average 7.7 turns over 842 tasks and include unanswerable turns; the file reports no per-turn numbers for MTRAG, so we will measure our own baselines there. We use CORAL, with retrieval recall and citation precision; the reported reference points are KD-ANCE at Recall@20 40.3% and summarization at 31.1% citation precision. We use the MuMu-QA long-dialogue split, against CMT-RAG's 41.73 EM. We use RECOR, where history plus reasoning reaches nDCG@10 .479 and DIVER reaches .584. The RECOR test targets the retrieval read path only.
The first decisive check is contamination persistence. On each benchmark we create a noisy variant. At a chosen turn $k$, we insert into the retrieved set either up to 25 distractor chunks, following the Ext2Gen protocol, or a single fluent passage that asserts a wrong attribute of the entity in focus. Some of these passages are written about a confusable sibling entity, in the MES-RAG style. We then measure the fraction of later turns whose answer repeats the injected value, and the number of turns until the system stops repeating it. Our prediction is that full history, summarization, overwrite, and convex averaging all show long persistence, and that gated memory confines the error to turn $k$.
The second check measures accuracy and citation precision by turn index, so that we can see whether the drop after turns four to six flattens.
The third check tests the inertia trade-off. We compare last-turn-plus-memory against full history on accuracy, repetition rate, and prefill time. The target is to keep the clipping speedup the inertia paper reports while matching full-history accuracy on questions that depend on turn one.
Finally, we take emerging nicknames from the DynamicER benchmark and insert them into synthetic dialogues. This tests whether linking errors, which are the main failure mode the gate cannot catch, are what bounds the remaining errors. If most residual errors trace back to wrong links rather than wrong writes, the next step is to add the pending-slot logic to mention linking itself.
References
- Dynamic Entity Representations in Neural Language Models
- Incremental Neural Coreference Resolution in Constant Memory
- DynamicER: Resolving Emerging Mentions to Dynamic Entities for RAG
- GAM-RAG: Gain-Adaptive Memory for Evolving Retrieval in Retrieval-Augmented Generation
- What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue
- NE-Table: A Neural key-value table for Named Entities
- MES-RAG: Bringing Multi-modal, Entity-Storage, and Secure Enhancements to RAG
- Mitigating Conversational Inertia in Multi-Turn Agents
- CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmented Generation
- CMT-RAG: Complementary Memory Traces for Multi-turn Multi-hop RAG
- Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
- mtRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems
- RECOR: Reasoning-focused Multi-turn Conversational Retrieval Benchmark