Context Compression via Recursive State Summarization

286 concepts203 papers3 proposals

When Large Language Models (LLMs) engage in long-horizon tasks—such as multi-session dialogues, complex multi-agent debates, or multi-step GUI navigation—they inevitably run into the limits of their context windows. Even as maximum token limits expand, appending raw interaction histories introduces a heavy computational tax due to the quadratic complexity of self-attention. More importantly, raw history is inherently noisy. Informal filler, redundant agent monologues, and failed intermediate reasoning steps dilute the prompt, causing models to lose track of critical information.

To solve this, a major research direction has emerged: recursive state summarization and context compression. Instead of treating history as an ever-growing log of raw tokens, these techniques treat history as a dynamic, compressed state that updates recursively over time.


History as a Sufficient Statistic

The core intuition behind state-based compression is that a well-constructed summary or belief state can act as a sufficient statistic for the entire past interaction. It contains all the necessary information to inform the next step, allowing the system to safely discard the raw token history.

Naive Approach (Linear Growth & High Noise):
[ Turn 1 ] -> [ Turn 2 ] -> [ Turn 3 ] -> [ Turn 4 ] -> Predict Next Turn

State-Based Approach (Bounded Context & High Signal):
[ State 1 + Turn 2 ] -> Summarize -> [ State 2 ]
[ State 2 + Turn 3 ] -> Summarize -> [ State 3 ]
[ State 3 + Turn 4 ] -> Predict Next Turn

This mechanism is highly effective in task-oriented dialogue systems. In Dialogue State Tracking (DST), Sungdong Kim et al. and Su Zhu et al. demonstrated that the previously predicted dialogue state can serve as a complete, structured summary of past interactions. By concatenating only the current utterance with this persistent, compressed state memory, models avoid the redundant processing of long dialogue sequences, significantly reducing input sequence length and computational overhead without sacrificing tracking accuracy.

Beyond structured slots, this principle applies to open-ended agent environments. Aly Lidayan demonstrated that compressing full interaction histories into concise natural language "belief states" can slash peak token usage by nearly half while maintaining or even improving task performance.


Recursive Compression and Memory Banks

When interactions span days, weeks, or months, maintaining context requires a multi-tiered or recursive distillation strategy to filter out conversational noise and preserve salient facts.

Arthur Szlam et al. pioneered a "read-write" memory strategy for long-term open-domain chat. Instead of storing raw dialogue logs, an encoder-decoder model generates concise summaries of salient personal facts and events from current interactions to be stored in a memory bank. This increases information density and reduces noise, making subsequent retrieval of relevant context more efficient and accurate.

To scale this over extended periods, systems perform periodic introspection:

  • Temporal Distillation: Wanjun Zhong et al. developed a framework that periodically distills raw dialogue logs into daily event summaries and high-level personality assessments, ensuring long-term user tracking fits within standard prompt limits.
  • Trait Introspection: In human-robot interaction, Hongtao Li et al. use periodic introspection to condense a history of specific user corrections into high-level personality traits, maintaining personalized preferences while reducing memory overhead.

Hybrid and Hierarchical Context Architectures

A common failure mode of pure summarization is the loss of immediate local context. If a history is compressed too aggressively, the model may lose the exact phrasing or immediate nuances required to generate a natural response.

To balance global continuity with local precision, researchers utilize hybrid architectures. Vishakh Padmakumar et al. showed that a hybrid approach—combining an abstractive summary of the distant past with the raw text of the most recent utterances—consistently outperforms simple sequence concatenation in long-form conversations.

Similarly, Yushi Hu et al. represent dialogue history as a combination of the most recent state summary and the current turn's utterances. This recursive representation minimizes token usage while preserving the immediate conversational context.


Beyond Dialogue: Agents, Planning, and Reasoning

The utility of state-based compression extends far beyond text dialogue into complex operational and reasoning tasks where long execution logs threaten to overwhelm the model.

GUI and Action Navigation

In multi-step graphical user interface (GUI) automation, maintaining raw visual or action histories quickly saturates context windows. Tao Liu et al. replace raw visual history with a compact, updated textual summary of previous interactions. To manage long input contexts in multi-step navigation, Zhiyuan Hu et al. generate concise natural language summaries of previous actions and screen states, compressing the trajectory history into a manageable representation while retaining critical context for future decision-making.

Multi-Agent Collaboration

When multiple agents debate or collaborate over multiple rounds, token counts explode. Ziyang Xiao et al. and Yilun Du et al. address this by using context-efficient communication protocols. By summarizing the collective insights or responses of multiple agents before passing them to subsequent rounds, they prevent context window saturation while preserving the essential arguments of the debate.

Long-Form Reasoning Traces

For heavy "Chain-of-Thought" reasoning models, internal monologues can stretch into tens of thousands of tokens, causing inference speed to crawl. Zhicheng Yang et al. mitigate this by alternating between verbose internal monologue and condensed summaries. This mechanism allows the model to perform deep local computation while maintaining a global logical state through a compact history of previous conclusions.


Why It Works: The Information Bottleneck

Why does an LLM often perform better when fed a summary rather than the full, raw history? The answer lies in the information bottleneck and the mechanics of attention weights.

  1. Maximizing Information Density: Raw text has low information density. Informal dialogue, repetitive reasoning steps, and redundant logs introduce token-level noise. Summarization forces the system to extract semantic invariants (e.g., facts, states, goals) and discard the filler.
  2. Mitigating Attention Dilution: Transformers distribute attention weights across all input tokens. When a prompt is bloated with irrelevant historical details, the attention mechanism can suffer from dilution or "forgetting" critical details in the middle of the context. A compressed state focuses the model's attention weights directly on high-signal transitions.
  3. Linearizing Computational Complexity: As demonstrated by Jia-Nan Li et al., transitioning from token-level caching to turn-level or state-level compression reduces the computational complexity of long-context dialogue from quadratic to linear.

By treating context management as an active process of recursive distillation rather than passive accumulation, these frameworks allow LLMs to maintain long-term coherence, scale to complex multi-step tasks, and operate efficiently over indefinite horizons.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_dialogue_history_summarization_b62f05b9,
  title = {dialogue history summarization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dialogue-history-summarization/rdr_b62f05b990674e958acc9bc9a82304fe},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_dialogue_history_summarization_b62f05b9_v1,
  title = {dialogue history summarization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dialogue-history-summarization/rdr_b62f05b990674e958acc9bc9a82304fe/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake