Programs that read long documents often keep a small memory of the people and objects mentioned so far. Each entry is one entity (a character, a company, a place) stored as a vector that is updated whenever that entity is mentioned again. When this memory has a fixed size, something must decide which entity to throw out. Today that decision is a rule of thumb: drop whoever has been silent longest, or whoever was mentioned least recently and least often. These rules treat a protagonist who has been off-stage for two pages the same as a waiter mentioned once. We propose training the eviction decision directly on the question it is trying to answer: how soon will this entity be mentioned again? The training labels cost nothing to produce. In any text with coreference annotation (coreference means marking which mentions refer to the same entity), every entity's future mentions are already written down, so for each entity at each moment we know the true time until it is next mentioned.
This turns entity eviction into a caching problem that has a known optimum. Belady's classical rule says to evict the item whose next use lies furthest in the future, and that rule can be computed exactly from gold annotations. The project measures how far existing heuristics fall below this oracle, trains a small predictor to imitate it, and tests whether the predictor closes the gap under tight memory budgets. The answer also settles a question that prior work left open: whether a learned eviction policy is worth having at all, or whether recency and frequency already capture almost everything.
Core Claim
Under a fixed budget of active entity slots, an eviction policy trained to predict each entity's time until next mention will keep coreference accuracy closer to the unbounded model than distance thresholds or salience decay. The advantage should appear where the two kinds of policy disagree: entities that are dormant but central, such as an absent protagonist, and entities that are recent but disposable, such as a just-introduced bystander. Concretely, let $g_e(t)$ be the number of tokens until entity $e$'s next mention after position $t$, with $g_e(t)=\infty$ if it never returns. The oracle evicts $\arg\max_e g_e(t)$. We train a predictor $\hat{g}_\theta$ that sees only information available at time $t$, and evict
$$ e^\ast = \arg\max_{e \in M_t} \hat{g}_\theta(e, t), $$
where $M_t$ is the set of active slots. The predictor is trained with a censored regression loss on log-gaps. Censoring is needed because the gap is unobserved past the document's end, and survival-style losses handle exactly this case. The first claim is empirical and can be falsified: the oracle must sit well above recency and frequency heuristics at small budgets. If it does not, there is nothing to learn, and we will report that.
Where This Sits
The mechanism we build on is incremental coreference with an entity bank. Incremental Neural Coreference Resolution in Constant Memory (also Revisiting Memory-Efficient Incremental Coreference Resolution) compares each new span against running entity vectors. It evicts singletons after 600 inactive tokens and multi-mention clusters after 1,200, reaching 79.4 average F1 on OntoNotes 5.0 against 79.6 for the full model while holding GPU memory at about 2.0 GB. The authors name a learned eviction policy as future work, and this proposal does exactly that, keeping their encoder and update rule unchanged. The Referential Reader is the other close precedent. It evicts through a Gumbel-softmax over inverse salience, a score that rises with mentions and decays over time, and it stays effective on GAP with as few as two cells. Its eviction is learned only indirectly, through the coreference loss. Its salience is purely a function of mention frequency and recency, which is the hypothesis we test against. Our policy is supervised directly by observed future mentions and may use the entity's content, not only its mention history.
Several papers motivate the problem without solving it. Towards Coherent and Consistent Use of Entities in Narrative Generation documents recency bias: plain language models drop protagonists who are absent for several paragraphs. Its entity-slot model improves attribute consistency by 57% relative on WikiPlots, but it pre-allocates slots for the characters in the opening prompt and never evicts. That makes it the natural setting for a second test in which slots must be recycled mid-story. On the pruning side, Darwinian Memory scores agent memory units by reuse, decay and failure, but it estimates value from repeated trials. Our hindsight labels avoid that cold-start cost. MEMORYLLM drops memory tokens at random, and TRELLIS learns forget gates over anonymous key-value vectors. Neither knows which stored item corresponds to which entity, which is the information our policy depends on.
One result constrains the scope. MEME shows memory systems collapsing on dependent updates, averaging 3% on Cascade and 1% on Absence. Better slot eviction does not address relational invalidation, so we make no claim about those tasks.
Research Program
The first stage is an oracle study and needs no new model. We take the released incremental coreference model and replace its token thresholds with a hard budget of $K$ active entities, sweeping $K$ over 4, 8, 16, 32 and 64. We run five policies at each budget: the Belady oracle computed from gold chains, least-recently-mentioned, Xia-style thresholds, salience decay in the style of the Referential Reader, and random. Running every policy over the same documents gives F1-versus-budget curves and their area. We also log which entities each policy evicts that the oracle keeps, broken down by mention type (named, nominal or pronominal), by mention count, and by subject frequency.
The second stage builds the predictor. It is a small network over the frozen entity vector plus cheap history features: mention count, tokens since last mention, the share of mentions that are named, and grammatical subject rate. It also cross-attends to the current segment's encoding, which is how it can notice cues such as a chapter break or a scene change. Training targets come from gold OntoNotes chains. For long texts, they come from silver chains produced by the unbounded model.
The third stage tests transfer to long text, where budgets actually bind. On NarrativeQA books, which average about 62,000 words, there is no gold coreference. We therefore measure fidelity: how well the budgeted model's clusters agree with the unbounded model's clusters on the same book.
The fourth stage, which is conditional on the first three succeeding, applies the policy to dynamically allocated slots in a D-MNEMELM-style generator on WikiPlots. Here the evicted entities are ones the generator would otherwise attend to.
Baselines
The naive reference points are the unbounded entity bank, which is the accuracy ceiling for a given encoder, and random eviction, which is the floor that MEMORYLLM adopts. The strongest existing method is the eviction policy of Incremental Neural Coreference Resolution in Constant Memory. Its reported OntoNotes 5.0 result is 79.4 average F1, against 79.6 unbounded, at a flat 2.0 GB. Those numbers come from token thresholds, not slot budgets, and we found no reported result on Lacuna for incremental coreference F1 under small fixed entity budgets. We will regenerate the baseline under our budgets with its own thresholds tuned per budget on the development set. The second strong competitor is salience decay as in The Referential Reader, which reports 72.1 GAP F1 (78.8 with BERT). We tune its decay rate per budget. Least-recently-mentioned eviction covers plain recency.
Every policy runs with the same frozen SpanBERT-large coreference weights, the same documents and the same budget. Each heuristic gets the same development-set tuning budget as the predictor's hyperparameters. The predictor adds well under 1% of inference compute, and we report wall-clock time to confirm this.
The key ablation is a history-only predictor. It is trained identically but sees only mention count and recency. This isolates whether content and context add anything beyond what a learned recency-frequency combination achieves.
Beating the baselines means two things. At every budget where the best tuned heuristic loses at least one F1 point against the unbounded model, the predictor must recover at least half of the heuristic-to-oracle gap. It must also beat the history-only ablation by a margin larger than the spread across random seeds.
Why not just…?
Why not just keep every entity, since entity vectors are tiny? This is a real threat for pure coreference. A thousand vectors cost almost nothing, and Xia et al. already move evicted entities to CPU memory. Our response is partly empirical and partly a matter of narrowing scope. Every span must be scored against every active entity, so compute and the number of distractor entities grow with bank size. It is possible that a large bank hurts accuracy by offering more wrong antecedents. The unbounded-versus-budgeted curves will show whether that happens. If unbounded banks dominate at negligible cost, we concede that eviction matters mainly where the active set is fed into attention, as in the generation setting of stage four, and we restrict the claim to that setting.
Why not just combine frequency and recency better? Mention count is probably the single best predictor of return, and a well-tuned salience score may already be close to the oracle. The history-only ablation is built to test exactly this. If it matches the full predictor, the honest conclusion is that a learned recency-frequency score suffices. That would still be a useful answer to the open question in Xia et al.
Why not just learn eviction end-to-end, as the Referential Reader does? End-to-end training only gets feedback when an eviction later causes a coreference error. That feedback is rare and arrives long after the decision. Hindsight labels give a direct target for every entity at every step. We will compare the two on GAP and OntoNotes at matched budgets.
Why not just retrieve from an archive of evicted entities when a mention seems unfamiliar? A two-tier memory, with dormant entities re-linked by nearest-neighbour search, is complementary to our policy rather than a rival to it. We will run it with and without the learned policy. The policy should still matter, because the oracle study tells us how often re-linking against a stale vector fails compared with keeping the entity active.
Decisive Checks
The project lives or dies on stage one. If the Belady oracle is not at least two F1 points above the best tuned heuristic at budgets of 8 to 16 slots on OntoNotes, or if the gap in cluster agreement with the unbounded model on NarrativeQA books is similarly small, we stop and publish the negative result. If the gap exists, the second check is whether the learned predictor closes at least half of it, whether it beats the history-only ablation, and whether its gains concentrate on dormant-but-central entities as the error breakdown predicts. Only then do we move to WikiPlots. There, success means attribute consistency under dynamic slot recycling that matches or exceeds the pre-allocated D-MNEMELM configuration, while handling entities introduced mid-story that the pre-allocated version cannot represent.
References
- Incremental Neural Coreference Resolution in Constant Memory
- Revisiting Memory-Efficient Incremental Coreference Resolution
- The Referential Reader: A Recurrent Entity Network for Anaphora Resolution
- Towards Coherent and Consistent Use of Entities in Narrative Generation
- Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolution
- MEMORYLLM: Towards Self-Updatable Large Language Models
- TRELLIS: Learning to Compress Key-Value Memory in Attention Models
- MEME: Multi-Entity and Evolving Memory Evaluation