EASE-KV: Diagnosing the Gap Between Memory Availability and Effective Use

Di Lu, Zechen Li, Xuanyu Yue, Jiabo Fan, Hao Xue, Flora D. SalimarXiv 2026 · arXiv

TL;DR

EASE-KV is a diagnostic framework that identifies why Large Language Models (LLMs) often fail to use "reusable" memory effectively—a phenomenon the authors call "reuse without recall." By decomposing the memory pipeline into Encoding, Allocation, and Supervision, the researchers show that effective memory requires more than just successful retrieval; it demands context-aware encoding and precise control over how different memories compete for the model's attention.

What Problem Does This Paper Address?

To build long-term AI assistants, we can't just keep stuffing the entire conversation history into the prompt; it's too slow and expensive. A popular alternative is reusable Key-Value (KV) memory, where past interactions are encoded once and their internal mathematical states (KV caches) are "injected" into the model during future queries.

However, there is a persistent performance gap: even when the correct information is present in the cache, the model often fails to use it to answer correctly. This happens because KV injection freezes representations before the query is known. Unlike standard text retrieval, where the model sees all facts together during a "prefill" stage, KV injection combines states that might have been encoded in isolation. This separates the moment a relationship between facts is formed from the moment it needs to be used.

Methods: The EASE-KV Framework

The authors introduce EASE-KV (Encoding–Allocation–Supervision Evaluation) to isolate where this pipeline breaks down.

Figure 1: EASE-KV asks three sequential questions under matched candidates and a frozen model: what is encoded, what can be allocated, and what can be learned from outcome supervision.
  1. Encoding Fidelity: Does the way we save memories preserve the necessary relationships? They compare Independent encoding (each fact alone), Block-diagonal (facts in one sequence but masked from each other), and Causal encoding (facts can see previous ones).
  2. Allocation Headroom: Can we "steer" the model to the right memory without re-encoding it? They implement signed attention-logit biases—essentially a volume knob for each memory—to see if simply weighting the "correct" memory higher fixes the answer.
  3. Supervision Resolution: Can the model learn a good allocation policy from feedback? They analyze whether simple binary rewards (correct/incorrect) provide enough "contrast" to teach a policy which memories are actually helpful.

Datasets

The study uses several benchmarks to stress-test different failure modes:

  • Controlled Compositional Setting: A custom set of atomic memories designed to isolate multi-hop reasoning (where Fact A + Fact B = Answer).
  • LongMemEval: A benchmark for long-term interactive memory in chat assistants.
  • Full-Haystack: A variant of LongMemEval that adds many irrelevant "distractor" memories to test the model's ability to ignore noise.
  • Temporal Conflict: A setting where the model must choose between an old fact and a newer, updated version of that fact.

Key Insights and Ideas

  • "Causal" Encoding is the Secret Sauce: If you encode facts in total isolation (Independent), you lose the "connective tissue" (like coreference or semantic links) needed for reasoning. Causal encoding, which allows later facts to attend to earlier ones during the caching process, is significantly more effective for multi-hop tasks.
  • Allocation is a Powerful Lever: You can significantly improve performance just by tweaking the attention weights of existing KV states. This proves that the information is often there in the cache, just underutilized.
  • The Supervision Bottleneck: Binary "correct/incorrect" feedback is a weak signal. In many cases, different memory allocations lead to the same binary outcome, leaving the learner with no idea which choice was better. "Shaped" rewards that provide more granular feedback are essential for training effective memory agents.
  • The Text-Latent Gap: There is a fundamental "delivery gap" between providing information as raw text versus injected KV states. Even with perfect allocation, injected states often lag behind their text counterparts because of the context lost during the initial encoding.

Key Results

The authors provide headline evidence across all three stages of their framework:

Figure 1: EASE-KV asks three sequential questions under matched candidates and a frozen model: what is encoded, what can be allocated, and what can be learned from outcome supervision.
  • Encoding: Causal encoding improved compositional accuracy by 25.0 percentage points over block-diagonal encoding. This highlights that "visibility" during encoding is a primary constraint.
  • Allocation: On the LongMemEval benchmark, simply using an "oracle" to weight the correct evidence higher (Evidence Allocation) boosted accuracy from 31.2% to 38.4% without changing a single bit of the stored memory.
  • Supervision: Moving from binary rewards to shaped rewards increased the fraction of "informative" query groups (those providing a clear learning signal) from 53.0% to 73.6%.
  • Temporal Recovery: In the temporal conflict task, hand-specified recency biases recovered 93.3% of the gap between latent memory and raw text, showing that the model can handle updates if told which memory to prioritize.

Limitations and Future Work

The authors acknowledge several constraints:

  • Fixed Components: The study uses fixed retrieved candidates and a frozen language model. In a real-world system, one might want to jointly optimize the retriever and the allocator.
  • Diagnostic vs. Deployable: Many of the best results (like the +7.2 pp gain in allocation) used "oracle" information—knowing the right answer in advance to test the system's limits. Turning these diagnostic insights into a deployable, autonomous policy remains a challenge.
  • Model Specificity: While they tested across Qwen, Llama, and Gemma, results may vary with different architectures or memory distributions.

Future work suggests focusing on "context-reuse trade-offs"—finding the sweet spot where we encode enough context to be useful but keep units small enough to be efficiently reusable.

Summary updated

Abstract

Reusable key-value (KV) memory avoids repeatedly prefilling retrieved text, but stored and injected memories do not necessarily become usable evidence. We present EASE-KV, a controlled framework for understanding this gap under fixed candidates and a frozen language model. EASE-KV separates three interfaces: whether required relations are preserved during encoding, whether preserved memories receive appropriate influence during allocation, and whether outcome supervision can identify useful allocations. Matched experiments show that causal encoding improves compositional accuracy by 25.0 percentage points over block-diagonal encoding, while the tested inference-time bridging methods fail to recover the lost contextual information. With unchanged stored states, diagnostic evidence-guided allocation improves LongMemEval accuracy from 31.2% to 38.4%, demonstrating that memory influence can be controlled without re-encoding. However, controllability does not guarantee learnability: binary correctness provides informative reward contrast for 53.0% of sampled query groups, compared with 73.6% under the evaluated shaped-reward setting. These findings show that effective KV memory requires more than retrieval and injection: the required relations must be encoded, preserved memories must receive appropriate influence, and supervision must distinguish better allocations from worse ones.
BibTeX
@misc{lu2026ease,
  author = {Di Lu and Zechen Li and Xuanyu Yue and Jiabo Fan and Hao Xue and Flora D. Salim},
  title = {{EASE-KV: Diagnosing the Gap Between Memory Availability and Effective Use}},
  year = {2026},
  eprint = {2611.11299},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11299},
}
Figure