Do Compressed Conversation Memories Know Which Facts Are Out of Date?

Chat assistants that run for hundreds of turns cannot keep the whole transcript in the prompt. One response is soft context compression: a small learned module squeezes long text into a handful of continuous vectors, called soft tokens, that the language model reads in place of the original words. These compressors are trained to preserve content. Conversations, however, also contain change. A user's budget is $600 in March and $1,200 in May, and anything computed from the budget changes with it. Explicit-memory systems show that models already cling to stale facts even when the correct update is in plain text. We propose to test whether soft compression makes this worse, and to repair it if so. The specific question is whether compressed vectors keep the order that tells an old value from its replacement, and not only the facts themselves.

The work has two parts. The first is a procedural generator of multi-session dialogues built from a hidden script of variables, updates, and dependencies. Because every instance is newly sampled, the answer key is exact and the test cannot be memorized from leaked benchmark data. The second is a training objective for soft compressors that explicitly penalizes answering with a superseded value. The deliverable is a measurement and a fix. The measurement shows how a "stale-answer rate" grows with compression ratio and with how often the old value was repeated. The fix is checked for transfer to existing static benchmarks.

Core Claim

Current soft compressors are trained with reconstruction or downstream question-answering losses. Those losses reward keeping a fact but never reward keeping the fact that it was replaced. We therefore predict that as compression tightens, errors shift toward a particular kind: the model answers with a value that was once true, and the shift is strongest when the old value was mentioned often. We also claim this failure can be fixed in the compressor, without adding a symbolic state store. The fix trains on generated dialogues where the stale value is used as an explicit negative. A loss that does this, given compressed memory $m$, a question $q$, the current value $v^{+}$, and a superseded value $v^{-}$, is

$$ \mathcal{L} = -\log p(v^{+}\mid m,q) + \lambda \max\big(0,\ \delta + \log p(v^{-}\mid m,q) - \log p(v^{+}\mid m,q)\big), $$

where $\lambda$ weights the margin term and $\delta$ is the required log-likelihood margin. The first term is the usual answer loss. The second term is new: it forces the compressed memory to rank the replacement above the value it replaced.

Where This Sits

The motivating evidence comes from explicit-memory work on conversational state. Can Agent Memory Systems Track Evolving State? calls the failure "state drift" and attributes up to 63.5% of confirmed errors to it in some tasks. It shows that drift persists even with oracle context and that chain-of-thought can make it worse. Its remedy, StateMem, stores discrete state units in a dependency graph and checks them with a deterministic rechecker. ACR documents "contextual inertia" and responds with discrete rewriting operators. Beyond Dialogue Time (TSM) shows that memories should be indexed by when an event happened rather than when it was mentioned. All three keep memory as text or symbols. None asks what happens to update ordering inside continuous vectors, which is our question.

The method we build on comes from soft compression. OSCAR compresses retrieved passages into memory tokens conditioned on the query and loses about 6% accuracy when the compression ignores the query. Pretraining Context Compressor (PCC) pretrains a compressor that a frozen model can use and reports sharp degradation beyond 16x to 64x. Both work in single-turn settings. The key constraint is C-DIC. It shows that repeatedly re-compressing earlier compressed states makes perplexity explode by 1,900%, and it avoids this with per-thread memory slots that are retrieved, revised, and written back. We adopt C-DIC's thread-slot design so our compressor does not inherit that collapse. Our contribution is the training signal, not the architecture. C-DIC was evaluated on MSC and MSC-QA, which contain no controlled updates.

The generator borrows its logic from dynamic contamination-resistant evaluation. DyVal generates reasoning problems from random directed acyclic graphs and finds that models strong on static benchmarks score near zero on them. INVITE builds fresh false-premise questions by distorting knowledge triples. SUMIE synthesizes entities whose attributes evolve and conflict, but only as documents to summarize. LongMemEval and StateMemBench are realistic dialogue memory tests, yet they are fixed releases and will eventually leak into training data. Our generator applies DyVal's graph-sampling idea to multi-session dialogue with scripted updates. It takes from LongMemEval the practice of hiding evidence among distractor sessions.

Research Program

The script. Each instance starts as a hidden "world script," sampled symbolically. The script contains a set of variables, such as a budget, a deadline, or a medication dose. Each variable has a value history with update events. Each event carries two times: the semantic time when it happened, and the session in which it is mentioned, following TSM's distinction. Dependency edges in the style of StateMem's derived_from link derived variables to base variables, for example remaining budget computed from budget and spending. Several controls are set independently: the number of updates per variable, the dependency depth, and the gap between an update and the question. The most important control is salience: how many times the stale value is restated before it is superseded. Entities, numbers, and domains are freshly sampled for each instance, so no item can be recalled from pretraining.

Rendering the script as dialogue. The script is turned into conversation by two separate agents: a user simulator holding the script and a persona, and an assistant. This split follows the patient/assistant decoupling of Zhang et al. and the personality-conditioned simulated patients of Bao et al., both of which aim to avoid overly cooperative, homogeneous self-chat. Whole sessions are generated at once, as Bae et al. recommend.

A verifier then extracts each scripted fact from the rendered turns and rejects any session where a fact is missing, contradicted, or stated in the wrong session. This keeps the answer key exactly equal to the script. Generation errors that pile up over long conversations are a known risk in synthetic dialogue, and this check addresses them.

Grading. Questions are graded against a closed pool containing the current value, every superseded value, and plausible distractors. This lets us separate stale answers from other errors, as StateMemBench does.

The compressor. We use C-DIC-style thread slots on a frozen backbone. The soft tokens are produced by a compressor initialized either from the backbone's first layers, in the style of OSCAR, or from a PCC-style pretrained compressor. It is trained on generated dialogues with the loss above, where $v^{-}$ is sampled from each variable's superseded history. We then run a linear probe on the soft tokens to test whether "which value is current" is linearly decodable. The probe distinguishes two explanations: that ordering information is destroyed by compression, or that it is kept but not used.

Baselines

The naive reference point is full prompting: the whole transcript stays in context. It is run on the same backbone up to its memory limit and gives the ceiling for accuracy without compression. C-DIC found that full prompting goes out of memory after 30–40 turns and scored 4.2% on MSC-QA. We report how its accuracy and latency change as length grows, rather than assuming it is an upper bound everywhere, because LongMemEval reports 30–60% drops for long-context models relative to oracle evidence.

The strongest soft-compression competitor is C-DIC itself, which reports 10.4% on MSC-QA. We retrain it on exactly the generated dialogues our method sees and give it the same number of slots and soft tokens. It is the right comparison because it shares our architecture and differs only in objective. We also train a query-conditioned OSCAR-style compressor and a PCC-style compressor on the same data.

The strongest method overall on state tracking is StateMem with StateMemWrapper. On StateMemBench it raised DeepSeek-V4-Flash current-state accuracy from 0.205 to 0.363, and the wrapper alone gave +32 to +67 points across six memory backends. For broader memory accuracy, TSM reports 74.80% on LongMemEval_S. We run StateMem on the same backbone and the same generated and public data. We report its accuracy alongside the tokens it reads, its per-turn extraction calls, and its wall-clock latency.

We found no reported result on Lacuna for any soft compressor on LongMemEval or StateMemBench. Those soft-compression numbers will therefore be produced here under a shared protocol.

Beating C-DIC means a lower stale-answer rate and higher current-state accuracy at equal soft-token budget and latency, on held-out generator configurations and on StateMemBench. Beating StateMem is the harder claim. It means matching its current-state accuracy within noise at substantially lower latency or token cost. If we come close but fall short, we report the accuracy–latency frontier honestly.

The ablation that isolates our ingredient trains the identical compressor on identical generated dialogues with the margin term removed ($\lambda = 0$). A second ablation trains with the margin term but replaces generated data with LongMemEval-style static knowledge-update examples. This tells us whether the generator's control over salience matters.

Why not just…?

Why not just use StateMem, since structured text state is already short? This is the real threat. A schema of current values may be only a few hundred tokens, which would make soft compression pointless for the state-tracking part of memory. StateMem's advantage, however, depends on extracting structure at every turn with a language model and on a schema that fits the domain. Open conversation contains far more than schema variables. The experiment that settles this compares, on matched backbones, StateMem's end-to-end latency and accuracy against our compressor's across the generator's dependency-depth and salience sweeps. If StateMem dominates at every operating point, we will report that supersession is better handled symbolically. The paper then becomes a diagnostic of soft compressors rather than a method paper.

Why not just keep a rolling text summary of the same length? Text summaries are inspectable, and that is a real advantage. SUMIE's human evaluation, though, found 45% redundant values and 25% unsupported attribute-value pairs in LLM incremental summaries. ACR's discrete rewriting relies on a trained router and multiple operators. We include a summary baseline whose token budget equals the soft-token count and that uses the same backbone. If it matches our stale-answer rate, the case for soft memory reduces to speed alone, and we will say so.

Why not just fine-tune a compressor on existing knowledge-update benchmarks? LongMemEval's knowledge-update questions are a small slice of 500 static items. They also do not let us vary salience or dependency depth, the two factors we hypothesize drive the failure. The static-data ablation tests this directly. If it transfers to StateMemBench as well as generator-trained models, the generator is useful only as an evaluation tool, not as a source of training data.

Why not just paraphrase or hold out a fixed released benchmark to avoid contamination? Rethinking Benchmark and Contamination with Rephrased Samples and On the Fragility of Benchmark Contamination Detection both show that rephrasing does not remove memorized shortcuts. We measure this ourselves. We freeze one generated set, publish it, and track how the gap between it and freshly sampled sets grows for models released afterward.

Decisive Checks

The first check is diagnostic. Among errors made by C-DIC-, OSCAR-, and PCC-style compressors trained without the margin term, the fraction that equal a superseded value should rise with both compression ratio (4x to 32x) and salience. This rise should be steeper than the fall in overall accuracy. If stale answers are simply a constant share of errors, the core claim is false.

The second check is the fix. With the margin term, the stale-answer rate should fall on generator configurations held out by domain, depth, and salience. Accuracy on current values should improve on StateMemBench and on LongMemEval_S knowledge-update and temporal-reasoning questions, without loss on MSC-QA or LoCoMo. The $\lambda = 0$ ablation should not reproduce these gains.

The third check is mechanistic. Linear probes should show whether recency becomes decodable from the soft tokens only after margin training, or was decodable before and simply unused.

All results are reported alongside latency and peak memory. These are the only grounds on which soft memory can justify itself against structured text state.