Spend Refinement Passes Only on "the Clerk": Per-Mention Halting for Role-Heavy Coreference

Programs that track people through a story mostly work by matching capitalized names. They fail on the people a story refers to only by role, such as "the coachman", "her father", or "that guy", and in conversation these role and pronoun mentions are the large majority. The strongest neural answer is iterative refinement: predict which mentions refer to the same person, feed that prediction back in, and correct it over several passes. That repair loop costs something, because every pass reprocesses every mention. The easy ones, like "Elizabeth Bennet" appearing for the fortieth time, get the same four passes as the genuinely ambiguous "the clerk" in chapter nine.

We propose a coreference model (coreference: grouping all mentions that refer to the same entity) that decides separately for each mention when to stop refining. Mentions that settle are frozen. Frozen mentions are summarized into fixed entity profiles, and later passes run only over the mentions still in doubt. We expect two consequences, and both can be tested. Latency should scale with the number of ambiguous mentions in a document rather than the total. Freezing should also curb the "drift" that iterative systems show, where repeated passes gradually merge two distinct background characters who share a job title.

Core Claim

The closest existing work is Graph Refinement for Coreference Resolution. It predicts a coreference graph over the whole document at once, feeds that graph back as input for later passes, and reports that accuracy peaks at a fixed four iterations with 80.5 average F1 on CoNLL-2012. Every mention there receives all four passes, the method has no early-exit mechanism, and it is evaluated only on general news and broadcast text.

Our claim has three parts. On role-heavy text, the benefit of extra refinement passes is concentrated in a minority of mentions, mostly role descriptors and distant pronouns. A learned, per-mention stopping rule can recover the accuracy of uniform four-pass refinement at a fraction of the wall-clock cost. And freezing resolved clusters reduces over-merging of same-title actors compared with running uniform refinement for longer. We are not proposing adaptive computation in general. We are proposing the specific claim that refinement effort should follow referential ambiguity, and that role mentions are where the ambiguity lives.

Why This Intersection

Evidence on the role side motivates the proposal. Joint Coreference Resolution and Character Linking for Multiparty Conversation finds that named entities make up only 12% of mentions in Friends transcripts. Its joint model lets mentions attend to each other within a scene and reaches 85.06 coreference F1 and 87.0 character-linking F1. It is single-pass and scene-bounded, so it neither corrects itself nor scales to book-length text.

Mr. Bennet, his coachman, and the Archbishop walk into a bar... builds an eight-stage handcrafted pipeline for detecting literary characters. It uses gender and conjunction "anti-edges" to stop distinct clerks from merging, and "behavioral bootstrapping" to promote common nouns that perform sentient actions to character status. It reaches 0.75 F1 on Pride and Prejudice, against 0.50 for BookNLP. Its later stages cannot repair failures from earlier ones, and it breaks on first-person narratives. We take its constraints as the target behavior that a learned model should reproduce without hand rules.

Semantic Role Labeling with Iterative Structure Refinement supplies the method template. A cheap first predictor is followed by a small refinement network that repairs global inconsistencies, which lifts English CoNLL-2009 labeled F1 to 90.99. It operates within sentences and never measures runtime.

The latency literature both motivates and constrains the design. Volcano shows a critique-revise loop more than doubling response time, from 2.7 s to 5.8 s. Meta-Reflection shows that moving reflection into hidden states can cut first-token latency from 951 ms to 246 ms at comparable accuracy, which argues for non-verbal, in-network refinement. Teaching Language Models to Self-Improve by Learning from Language Feedback is a counterweight: ablating the verbal critique hurt its results. We address that tension directly with a verbal-critique comparison in the experiments. RepoCoder and Extrapolative Controlled Sequence Generation via Iterative Refinement both name stopping criteria and drift as unsolved problems. Our freezing rule is a concrete answer to them in a setting where drift can be measured: false merges.

Research Program

The base model follows the graph-refinement design. A pretrained encoder (SpanBERT-large, matching both reference papers) proposes candidate mentions. We then apply the "reduced document" idea from Graph Refinement for Coreference Resolution, which keeps only mention tokens, and a stack of refinement passes updates each mention's link distribution over earlier mentions and its own current prediction.

We add a small halting head. At each pass $t$, it reads a mention's representation, the change in its predicted antecedent distribution since pass $t-1$, and the margin between its top two antecedents, and it outputs a probability $h_i^t$ that mention $i$ is settled. Training minimizes

$$\mathcal{L} = \sum_{i}\sum_{t=1}^{T} q_i^t\,\ell_i^t + \lambda \sum_i \mathbb{E}[\tau_i],$$

where $\ell_i^t$ is the coreference loss for mention $i$ if it stops at pass $t$, $q_i^t$ is the probability of stopping exactly at $t$ (built from the halting probabilities), $\tau_i$ is its stopping pass, and $\lambda$ trades accuracy for computation. Sweeping $\lambda$ traces out an accuracy-latency curve rather than a single operating point.

A mention that halts is frozen. Once every mention of a cluster is frozen, the cluster is pooled into a fixed entity vector. Later passes let each active mention attend to the other active mentions and to the frozen entity vectors, but not to individual frozen mentions. The cost of a pass therefore grows with $A(A+E)$, where $A$ is the number of active mentions and $E$ the number of frozen entities, instead of $M^2$ over all $M$ mentions. This matters for novels, where $M$ runs into the tens of thousands.

We also forbid merges between two already-frozen clusters. An active mention can join a frozen cluster, but two settled clusters cannot be merged by drift. An ablation that lifts this rule measures what it buys.

Book-length inputs are processed in overlapping windows, and frozen entity vectors carry across windows. This gives the cross-scene tracking that the scene-bounded model of Joint Coreference Resolution and Character Linking lacks. For character detection, we convert clusters to a character list by keeping clusters that meet a size threshold and whose mentions act as subjects of sentient verbs. This is a learned-cluster version of Vala et al.'s behavioral bootstrapping step.

The analysis questions are where passes get spent (proper names, role nominals, pronouns), how that allocation changes with genre, and whether the halting head's confidence is calibrated, meaning that frozen mentions really are correct at the rate the head predicts.

Baselines

The naive reference point is a single pass of the same model with refinement disabled, run at one iteration. It anchors the latency floor and shows how much refinement buys at all on each benchmark.

The strongest existing method is uniform graph refinement at the fixed depth that Graph Refinement for Coreference Resolution found optimal, four passes. On CoNLL-2012 its reported number is 80.5 average F1 with SpanBERT-large. No reported result was found on Lacuna for its wall-clock latency, or for its performance on Friends or literary text. We will therefore re-run it ourselves with the same encoder, the same mention detector, the same data splits, the same GPU, and the same hyperparameter search budget as our model. We will report it at every fixed depth from one to six, which gives a uniform-depth accuracy-latency curve. Beating it means one of two things. Either our curve lies strictly above it, with higher F1 at equal measured latency, or we match its four-pass F1 within noise (seed standard deviation over five runs) at no more than half its measured latency. On Friends, the reported Joint Coreference Resolution and Character Linking results (85.06 coreference F1, 87.0 linking F1, SpanBERT-large) serve as the published bar. We will add its linking head to our model so the comparison covers both tasks.

For book-length character detection, the reference points are the reported 0.75 F1 on Pride and Prejudice and 0.54 on Sherlock Holmes from Mr. Bennet, his coachman..., and BookNLP at 0.50 from the same paper. Matching the handcrafted pipeline without its rules would already be informative. Exceeding it on the Sherlock Holmes corpus, where minor role-only characters dominate, is the target.

The ablation that isolates the key ingredient replaces the learned halting head with an orthographic rule. Capitalized proper-name mentions freeze after pass one, and every other mention runs the full four passes. This rule costs almost nothing to compute, so it is also the most dangerous competitor. It is treated in the next section.

Why not just…?

Why not just use two uniform passes instead of four? If the fixed-depth accuracy curve in Graph Refinement for Coreference Resolution is nearly flat after pass two, a uniform cut gives most of the savings with no new machinery. We cannot rule this out from the file, since only the location of the peak is reported, not the curve. The uniform-depth sweep in our baselines answers the question directly. If uniform two-pass refinement already sits on our curve, the adaptive method's value reduces to the drift and long-document benefits, and we will say so.

Why not just route by capitalization: freeze names, iterate on everything else? This is the real threat. The motivating statistic, 12% named mentions in Friends, is also the case for the heuristic. We expect it to fail in both directions. Some role mentions are trivial, such as a unique "the Archbishop" or a pronoun right after its antecedent. Some names are hard, such as the alias splits like "Miss Tilney" versus "Eleanor Tilney" documented in Evaluating Computational Representations of Character. Because most mentions in dialogue are pronouns, the rule would also iterate on most of the document. We will compare the learned head and the rule at matched latency and break down accuracy by mention type. If the rule matches the learned head, the right conclusion is that a cheap orthographic router suffices, and that becomes the paper's finding.

Why not just prompt a large language model to cluster the characters in one pass with reasoning? For short scenes this may be competitive, and we will run a GPT-4-class model on the Friends test scenes with the same mention spans supplied. The file contains no reported coreference score for such prompting on these benchmarks. The related evidence is mixed: GPT-4 with reasoning reaches 0.98 on Austen gender grouping but only about 50% expert agreement on character similarity. More basically, reasoning tokens scale with document length. The latency findings from Volcano and Teaching Language Models to Self-Improve point the wrong way for novels. We will report its F1 and per-scene latency alongside ours rather than assume either result.

Why not just add a verbal critique step, since critiques helped self-improvement? Teaching Language Models to Self-Improve found that critiques matter for open-ended generation. Coreference errors, by contrast, have a small, structured space of possible repairs: relink, split, or merge. We therefore expect in-network refinement to carry the same information more cheaply, consistent with Meta-Reflection. A small-scale check, in which a model writes critiques of disputed links on the Friends development set, will test whether critiques fix errors that our halting-refinement model leaves.

Proposed Experiments

The first experiment establishes the Pareto comparison on CoNLL-2012 and on the Character Identification V2.0 Friends data. We report MUC, B³ and CEAF average F1 against measured wall-clock time per document and total mention-pass count, for our model across $\lambda$ and for uniform refinement across depths. It passes if our model matches four-pass F1 at half the latency on Friends, where role mentions dominate. On CoNLL-2012 we expect a smaller gap, because news text is more name-heavy. A larger saving there than on Friends would contradict our core claim.

The second experiment tests where the computation goes. For each mention type, we report the average stopping pass and the F1 gain from refinement. The claim predicts that role nominals and distant pronouns receive most passes and account for most of the gain.

The third experiment measures drift. We build a small diagnostic set from the 58-work literary gold standard of Mr. Bennet, his coachman...: pairs of distinct characters sharing a title or common noun, such as two clerks or two servants. On this set we measure the false-merge rate for uniform refinement at depths four through eight, for our model, and for our model with the frozen-merge ban removed. The claim predicts that false merges rise with uniform depth and stay flat under freezing.

The fourth experiment scales to books. We run the windowed model with carried-over entity vectors over Pride and Prejudice and the Sherlock Holmes corpus, convert clusters to character lists, and score against the reported 0.75 and 0.54 F1. We also report how active-set size and runtime scale with book length, to check the predicted $A(A+E)$ cost against the $M^2$ cost of uniform refinement. Where the annotations of Towards Coreference for Literary Text are available, we use them qualitatively to examine failures on narrator shifts and generic-versus-specific nouns, since that paper reports no scores to compare against.