TL;DR
Large Language Models (LLMs) struggle with the massive memory costs of long-context windows. This paper introduces Addressable Context Encoding (ACE), a method that stores context directly into an LLM's weights via test-time updates, using unique identifiers and specialized prompting to ensure the model actually uses that information as active context rather than just "memorizing" it.
What Problem Does This Paper Address?
As we push LLMs to process entire books or massive codebases, we hit a wall: the "KV cache" (the model's short-term memory) grows linearly with context length, while the computation cost grows quadratically.
A promising alternative is Test-time Context Encoding (TCE), where we temporarily update the model's weights to "absorb" the context. However, previous TCE methods had a major flaw: the model often couldn't distinguish between its pre-existing general knowledge and the newly encoded context. It might "memorize" a fact (be able to recite it) but fail to "use" it (apply it to answer a question). Most existing solutions rely on expensive "meta-training" to teach the model how to do this, making them difficult to use with standard, off-the-shelf models.
Methods
The authors propose Addressable Context Encoding (ACE), which works with off-the-shelf LLMs without extra training. It relies on two key components:
- Addressable Mechanism (UIDs): During the encoding phase, each chunk of context is paired with a Unique Identifier (UID)—a random string like
ctx-573277c6. The model is updated to associate the context with this specific "address." - Prompting Framework:
- Role-Binding Prompt: During encoding, the model is told: "This text is a context chunk... identified by [UID]. It should be treated as conversational context."
- Context-Invocation Prompt: During inference, the model is prompted to "Recall context [UID] to continue." This explicitly tells the LLM to treat the information stored at that "address" as the active context for the current query.
By updating only specific "fast weights" (usually the final feed-forward network layer), the authors can inject new information without destroying the model's original capabilities.
Datasets
The researchers evaluated ACE using a mix of controlled "sandbox" tasks and established long-context benchmarks:
- Sandbox Tasks: Newly curated tests for Recall (retrieving a specific value), Recognition (identifying what was just told), Continuation (answering a pending question), and Conflict (handling information that contradicts the model's internal training data).
- RULER: A "Needle-in-a-Haystack" benchmark that tests retrieval from 1K to 64K tokens.
- LongBench & LongBench v2: Comprehensive benchmarks for long-document understanding and reasoning.
- Retention (MMLU): Used to ensure that encoding new context doesn't "break" the model's general knowledge.
Key Insights and Ideas
- Recitation $\neq$ Contextual Use: A model can be "fine-tuned" on a document until it can recite it word-for-word, yet still fail to use that document to answer a question. ACE bridges this gap by providing an explicit "access path" via UIDs.
- The "Address" is the Key: Using a UID acts like a database key. It provides a strong signal that helps the model's attention mechanism focus on the specific weight updates associated with that context.
- Off-the-Shelf Viability: Perhaps the most important finding is that you don't need complex meta-learning to make TCE work. With the right prompting and addressing strategy, standard models like Qwen3.5 or Llama-based models can effectively "learn" context at test-time.
Key Results
ACE showed dramatic improvements over standard TCE across almost every metric:
- Sandbox Performance: On the Qwen3.5-27B model, ACE increased Recall accuracy from a dismal 1.5% to 75.6% and Recognition from 0.0% to 98.0%.
- Handling Conflicts: ACE was remarkably good at prioritizing encoded context over pre-existing knowledge. In "Counterfactual" tasks (where the context lies to the model), ACE achieved over 90% accuracy in following the context, whereas standard TCE often defaulted back to its original training.
- Long-Context Benchmarks: On the RULER NIAH-3 task, ACE improved accuracy from 8% to 63% using the Muse-Glimmer-30B model.
Limitations and Future Work
- Computational Overhead: While ACE saves memory (KV cache), it requires backpropagation during the encoding phase. This is currently slower than simply filling a KV cache for shorter contexts.
- Reasoning Gap: While ACE is excellent at retrieval, it still lags behind "Gold Standard" (full-context) inference on complex reasoning tasks like LongBench v2.
- Scaling Updates: The authors noted that while updating more layers can help, it's a balancing act; too many updates can start to degrade the model's general reasoning (the "Retention" problem).
Future research will likely focus on making the weight updates more efficient and improving the model's ability to perform multi-step reasoning over encoded information.
Summary updated