Dynamic Memory Banks for Tracking Entity States

42 concepts30 papers3 proposals

When reading a narrative or analyzing a complex document, humans naturally maintain a mental catalog of the participants—characters, objects, or organizations—and track how their states change over time. If a character is introduced as an engineer in Chapter 1, disappears for several pages, and returns in Chapter 4, we instantly recall their traits.

Standard sequence models and Transformers struggle with this task. In a traditional architecture, all contextual information is compressed into a single hidden state stream or distributed across a flat self-attention window. As the context grows, specific details about individual entities become diluted or completely forgotten. Furthermore, tracking every past mention explicitly introduces quadratic computational complexity relative to the document length.

To solve this, a prominent research direction decouples general context modeling from entity tracking by introducing dynamic entity memory banks. Instead of relying on a flat token history, these architectures maintain a dedicated set of memory slots that explicitly track the evolving states of individual entities.

Decoupling Identity from State

The core mechanism of a dynamic entity memory bank relies on separating who an entity is from what is currently known about them. This is frequently implemented using a key-value paradigm.

As explored by Laura Aina et al. in What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue, these architectures utilize static keys to establish the unique identity of an individual, while dynamic values are updated as the narrative progresses. This structural bias allows the model to distinguish between different entities of the same type within a specific document—a capability also leveraged by Janarthanan Rajendran et al. in NE-Table: A Neural key-value table for Named Entities to handle rare or out-of-vocabulary entities without expanding the model's core vocabulary.

By mapping entities to specific memory slots, the model creates a stable framework akin to variables in a computer program. Haritz Puerto et al. demonstrate in Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs that explicitly assigning entities to variables significantly reduces memory errors during logical deduction. This isolation ensures that the internal dynamics and history of one object do not contaminate another over long time horizons, a principle fundamental to tracking moving objects or persistent discourse participants (Joshua B. Tenenbaum et al., Learning Online Data Association).

The Mechanics of Incremental Updates

How does information enter and evolve within these memory slots? The process operates sequentially from left to right. As the model processes text token-by-token, it identifies mentions and routes the local context vector to the corresponding entity's memory slot.

[Input Text] ---> "John walked into the room."
                        |
                (Mention Detected: John)
                        |
                        v
         [Routing Gating Mechanism]
                        |
         +--------------+--------------+
         |                             |
         v                             v
   [Slot 1: John]               [Slot 2: Room]
   Old State: [Vector]          Old State: [Vector]
   Update via Convex Comb.      Update via Convex Comb.
         |                             |
         v                             v
   New State: [In Room]         New State: [Contains John]

In early statistical systems, such as the multilingual framework developed by Chinatsu Aone et al. in A Language-Independent Anaphora Resolution System for Understanding Multilingual Texts, this tracking was managed monotonically: attributes were incrementally appended to persistent structures to guarantee that previously established context was never lost during subsequent references.

In modern neural architectures, this update process is continuous and differentiable. In the ENTITYNLM model introduced by Yangfeng Ji et al. in Dynamic Entity Representations in Neural Language Models, entity representations are maintained in a persistent memory bank and updated using a convex combination of the existing entity embedding and the current hidden state of the recurrent network.

To scale this process to long documents without an explosion in memory consumption, Patrick Xia et al. in Incremental Neural Coreference Resolution in Constant Memory show that representing entities as single, evolving vectors updated via a learned weighted average is an exceptionally effective alternative to storing every individual mention embedding. This bounded approach avoids the quadratic complexity of traditional mention-pair methods by managing active entities within a fixed set of memory cells (Fei Liu et al., The Referential Reader: A Recurrent Entity Network for Anaphora Resolution). As noted by Marta Recasens in Coreference Resolution across Corpora: Languages, Coding Schemes, and Preprocessing Information, such incremental, entity-based architectures allow the system to maintain a stateful representation across the entire document by continuously accumulating linguistic attributes.

Why It Works: Long-Range Coherence and Consistency

By maintaining dedicated, stateful vectors for individual entities, models achieve a dramatic improvement in long-range consistency across both understanding and generation tasks.

In narrative generation, standard language models often suffer from the "forgetful character" problem. Elizabeth Clark et al. address this in Neural Text Generation in Stories Using Entity Representations as Context by showing that updating vector memories upon each mention ensures characters behave consistently according to their accumulated history. Similarly, Pinelopi Papalampidi et al. demonstrate in Towards Coherent and Consistent Use of Entities in Narrative Generation that integrating a dynamic memory module into a transformer architecture allows for the persistent tracking of character states even when the original introduction context has fallen completely out of the model's primary attention window.

This stateful tracking also prevents redundancy and ensures structural coherence. In data-to-text generation, Ratish Puduppully et al. in Data-to-text Generation with Entity Modeling utilize a gated mechanism to update entity vectors during text generation. This allows the system to track exactly which facts about an entity have already been articulated, preventing the repetitive output of information.

Resolving Omitted and Evolving Context

A final major advantage of dynamic entity memory is its ability to resolve implicit or shifting context where local text clues are insufficient.

For instance, in languages like Japanese, arguments are frequently omitted when they can be inferred from discourse context (zero anaphora). Tomohide Shibata et al. demonstrate in Entity-Centric Joint Modeling of Japanese Coreference Resolution and Predicate Argument Structure Analysis that a persistent memory buffer allows a model to resolve these omitted arguments by comparing current phrases against the evolved state of all known entities introduced several sentences prior.

Text Stream:  "The company launched a new product [Entity A]. ... [3 Sentences Later] ... It was a massive success."
                                                                                       |
Flat Context:                                                                  "It" = ??? (Diluted)
                                                                                       |
Entity Memory Bank: [Slot A: Product State = Evolved Context] <------------------------+ (Resolved)

Furthermore, entities are not static in the real world; they acquire new descriptions and nicknames over time. Jin Young Kim et al. address this linguistic drift in DynamicER: Resolving Emerging Mentions to Dynamic Entities for RAG. By calculating a weighted average of a canonical knowledge base description and the embeddings of mentions linked to the entity in preceding periods, the semantic representation of the entity evolves dynamically. This allows retrieval and generation systems to maintain accuracy as real-world context shifts, bridging the gap between static knowledge bases and fluid, evolving discourse.

Key Foundational Work

A Language-Independent Anaphora Resolution System for Understanding Multilingual Texts (1993) — contributed 2 concept elements to this direction.
Reference Resolution beyond Coreference: a Conceptual Frame and its Application (1998) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_dynamic_entity_memory_a4e0f09f,
  title = {dynamic entity memory},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dynamic-entity-memory/rdr_a4e0f09fb0d64739a050f5cc6156f284},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_dynamic_entity_memory_a4e0f09f_v1,
  title = {dynamic entity memory},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dynamic-entity-memory/rdr_a4e0f09fb0d64739a050f5cc6156f284/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake