When large language models (LLMs) are tasked with summarizing long documents, multi-turn dialogues, or evolving news streams, the standard approach is to feed the entire text into the context window at once. However, this "all-at-once" paradigm suffers from severe structural flaws. As demonstrated by Wan et al. and Sun et al., LLMs exhibit a pronounced "lost-in-the-middle" or positional bias, frequently failing to faithfully capture or synthesize information located in the center of long inputs.
An alternative workaround is to split the text into isolated chunks, summarize them independently, and concatenate or hierarchically merge the results. But this introduces a new problem: it breaks narrative flow and destroys cross-chunk dependencies. As noted by Yue et al. and the authors of the ORCHID corpus, chunk-based methods often lead to factual errors and a loss of logical progression because the model cannot track how arguments or storylines evolve across boundaries.
Summarization as a State Machine
To solve these issues, a growing body of research shifts the paradigm from static, single-pass generation to incremental state-update summarization. Instead of treating summarization as a function that maps a massive text block to a single output, this approach treats it as a dynamic state machine.
Let $S_{t-1}$ represent the persistent summary state (or running summary) up to step $t-1$, and let $X_t$ represent a new incoming text segment (a document block, a dialogue turn, or a news update). The model updates the summary sequentially:
$$S_t = \text{Update}(S_{t-1}, X_t)$$
This formulation works because of three core intuitive mechanisms:
- Bypassing the Middle Curse: The model's active context window never holds the entire raw text history simultaneously. It only contains the compact previous summary $S_{t-1}$ and the immediate new segment $X_t$. This keeps the input size small and focused, eliminating positional bias.
- Structural Continuity: By conditioning the update on the prior summary, the model acts as an editor rather than a re-writer. Panthaplackel et al. show that incorporating the previous output as context significantly reduces unnecessary revisions and maintains structural continuity, minimizing the cognitive load for readers tracking evolving stories.
- Active Information Filtering: The running state acts as a lossy, compressed memory buffer. At each step, the model filters out noise from $X_t$ and integrates only the salient facts. Yoon et al. leverage this mechanism for active context compression in question answering, updating a persistent state segment-by-segment and stopping as soon as the summary contains enough information to answer the query.
Architectural Strategies for the Update Step
How exactly should an LLM execute the update function? Researchers have identified two primary strategies for combining the old state with new information:
- Direct Update (The UPDATE Method): The model receives $S_{t-1}$ and $X_t$ simultaneously and is prompted to output the revised summary $S_t$ in a single pass.
- Two-Step Merge (The MERGE Method): To prevent the model from overlooking fine-grained details in the new text, Hwang et al. propose a decoupled approach. The model first generates a micro-summary of the new segment $X_t$ in isolation, and then merges this new summary with the existing state table. The authors demonstrate that decomposing the task this way helps LLMs better handle incremental changes and resolve factual conflicts.
For highly complex interactions, such as multi-turn competitive debates, a simple text summary may not suffice. Liang et al. introduce a dual-memory system that explicitly separates raw incoming text from high-level analytical summaries. By feeding the model its own previous "digested" analysis alongside new data, the system tracks the logical evolution of arguments without losing focus to noise.
Applications Across Domains
The incremental state-update framework has proven highly versatile across several distinct domains:
- Long-Form Dialogue and Chatbots: In extended conversations, keeping the entire chat history quickly exhausts the context window. Lee et al. use incremental dialogue state summarization to periodically extract and store key facts into a long-term memory pool. Similarly, Duan et al. and Duan et al. employ an iterative summarization pipeline in autobiography interviewing, distilling individual sessions into concise contextual summaries to maintain coherence over long durations.
- Streaming Text and Evolving News: For continuously growing text streams, Chen et al. introduce stepwise summarization, where the model generates sequential summary snippets that must seamlessly integrate with prior history to build a cohesive, growing narrative.
- Book-Length Literature and Document Collections: When summarizing entire books, Chang et al. find that while hierarchical merging strategies produce highly coherent overviews, incremental state updating preserves the fine-grained narrative details that human readers actually prefer. However, Pratapa et al. caution that maintaining a persistent running summary across massive document collections heavily depends on the model's capacity to synthesize new facts without losing prior context over long horizons.
Historical Context and Trade-offs
While modern LLMs have supercharged this capability, the underlying intuition dates back to early symbolic AI. Lehnert originally proposed performing narrative summarization as an incremental process by traversing conceptual networks of interlocking mental states and events, allowing summaries to expand or contract while maintaining core narrative logic.
Today, the primary trade-off of incremental state updating is sequential latency versus global consistency. Because each step depends on the output of the previous step, the process cannot be parallelized like Map-Reduce approaches. Furthermore, as noted by Chang et al., the compounding complexity of repeatedly modifying a running state can occasionally introduce logical inconsistencies or factual drift if the model fails to properly reconcile conflicting information. Nonetheless, for preserving detail, maintaining narrative flow, and conquering the positional biases of long-context models, treating summarization as an incremental state-update process remains a powerful and intuitive paradigm.
Key Foundational Work
Updated Headline Generation: Creating Updated Summaries for Evolving News Stories (2022) — contributed 2 concept elements to this direction.
Narrative Text Summarization (1980) — contributed 1 concept elements to this direction.