Revisable Commitments for Two-Stage Anaphora Resolution in Dialogue

Many systems that resolve pronouns work in two steps. First, a detector decides whether a word such as "it", "that", "one" or "you" actually refers back to something; "it" in "it is raining" refers to nothing. Second, a resolver links each word that passed the detector to the earlier expression it refers to, which is called its antecedent. Splitting the work this way is cheap and modular. The cost is that the detector's decision is final. A missed anaphor can never be resolved, and a spurious one forces the resolver to invent a link. In conversation the situation is worse. Downstream components, such as a dialogue state tracker or a retrieval engine, consume these decisions turn by turn, so an early mistake keeps being acted upon. Yet the words that settle whether "that" pointed to a clause, or whether "it" was referential at all, often arrive one or two turns later.

This note proposes making the hand-off between the two stages revisable over time. The detector may mark a trigger as undecided instead of committing. The resolver reports back how well each trigger is supported by an antecedent as new turns arrive. A small set of repair operations on the entity memory can then detach, re-attach or re-type earlier mentions when the evidence changes. The claim is narrow and testable: a meaningful share of the end-to-end loss in dialogue anaphora comes from errors that later context would have fixed, and a cheap, targeted revision mechanism can recover most of what re-running the whole pipeline every turn would recover, at a fraction of the cost and with far less churn in what downstream components see.

Where the gap is

The size of the cascade is well documented. On the British National Corpus, the one-anaphora pipeline in Finding The Right One and Resolving it reports 83.78 F1 for detection and 67.38 F1 for resolution, but only 64.61 F1 when the two stages run end to end. In the CODI-CRAC 2022 dialogue benchmark, The CODI-CRAC 2022 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue evaluates the same systems three ways: with predicted mentions, with gold mentions, and with gold anaphors supplied. Bridging baselines average about 5% when they must predict their own anaphors and more than triple when gold anaphors are given. Discourse-deixis systems exceed 50% F1 with gold anaphors but the predicted-setting baseline sits near 15%. These gaps show that detection is the bottleneck. They do not say which detection errors are fixable, and that is the question this proposal is built around.

The existing two-stage designs share a feed-forward, commit-once interface, and they motivate the proposal rather than constrain it:

All of these decide once, in a single pass, with no channel from the resolver back to the detector.

The closest work to this proposal does one of two things. Some of it trains the stages jointly. Anaphora Resolution in Dialogue: System Description (CODI-CRAC 2022 Shared Task) trains anaphor-type classification together with antecedent selection and finds selection still bounded by the type classifier. Other work hardens the early decision. Coreference Augmentation for Multi-Domain Task-Oriented Dialogue State Tracking adds a "coref vs. none" slot classifier followed by span retrieval, reaching 56.47% Joint Goal Accuracy on MultiWOZ 2.1. Joint Goal Accuracy is the fraction of turns where the entire tracked state is correct. Improving Dialogue State Tracking with Turn-based Loss Function and Sequential Data Augmentation up-weights early-turn losses so that early mistakes are rarer. None of these lets a committed decision be undone at inference time; the turn-weighted loss paper says explicitly that it offers no repair once an error is made.

On the error-propagation side, two papers supply the mechanism this proposal borrows. LLM-as-RNN replaces append-only history with a rewritable memory and corrects 54.8% of past errors on MIMIC-IV. Diable treats dialogue state as a table edited by insert and delete operations. Both work with free text or slot tables, not coreference clusters. Diable also illustrates the danger this proposal must avoid: a missed edit persists forever because later turns only predict local differences.

Core Claim

Some detection errors in dialogue are recoverable by waiting, because the disambiguating evidence appears after the anaphor's turn. For this class of errors, three ingredients together lower both the final coreference error and the time that wrong links remain visible to downstream components:

  1. a three-way detector (anaphoric, non-anaphoric, undecided);
  2. a resolver that re-scores undecided and low-margin triggers against each new turn;
  3. explicit repair operations on the entity clusters.

If such errors are rare, the idea fails, and the first study is designed to find that out cheaply.

The revision signal is a single quantity. For a trigger $m$ at turn $t$, define its antecedent support as

$$s_t(m) = \max_{a \in C_t(m)} \mathrm{score}_t(m,a) - \mathrm{score}_t(m,\varnothing),$$

where:

  • $C_t(m)$ is the set of candidate antecedents available by turn $t$, including clausal spans for deixis;
  • $\mathrm{score}_t$ is the pairwise resolver score computed with a context window that now includes turns up to $t$;
  • $\varnothing$ is the "no antecedent" option.

A trigger is committed as referential when $s_t(m) > \delta$ and as non-referential when $s_t(m) < -\delta$. It stays pending otherwise, for at most $k$ turns. A previously committed decision is revised only when $s_t(m)$ crosses the opposite threshold. The margin $\delta$ is chosen on development data to control how often decisions flip.

Research Program

The first piece of work is diagnostic and needs no new model. On the CODI-CRAC development sets (Light, AMI, Persuasion, Switchboard), we train two detectors on the same encoder. One sees only the dialogue up to the current turn. The other also sees the next $k$ turns. We count the triggers that the prefix-only detector gets wrong and the look-ahead detector gets right, broken down by anaphor type: identity, discourse deixis, pleonastic "it", generic "you", and bridging. This count is an empirical ceiling on what revision can achieve.

We expect identity coreference of named entities to show almost no recoverable errors. We expect discourse deixis and pleonastic "it" to show the most, because a speaker's follow-up often makes clear whether "that" pointed at a proposition. If the recoverable share is a small fraction of the gap between the gold-anaphor and predicted settings, we stop and report it as a negative result. That result would itself tell the field that detection errors must be fixed at detection time.

If the ceiling is substantial, we build the revisable pipeline on top of a re-implementation of the Kim et al. architecture: start/end span scoring, pairwise scoring with speaker features, and an added null antecedent. The additions are a calibrated undecided band in the detector and a pending buffer. The entity memory supports three repair operations: detach a mention from its cluster, re-attach it to another cluster, and re-type it between referential and non-referential. When a new turn arrives, we re-encode only short windows that cover each pending or low-margin trigger together with the new turn. Encoder cost therefore scales with the number of uncertain triggers, not with dialogue length. History is kept unpruned for pending triggers. This is deliberate: context-compression methods such as ACR cut tokens by 83.4% on HotpotQA, but in dialogue anaphora they would discard exactly the distant clausal antecedents that deixis resolution needs, and CODI-CRAC already shows systems degrading on the long AMI meetings.

The second testbed tests whether revision matters to a real downstream consumer whose metric is itself a cascade. On MultiWOZ 2.1, using the coreference annotations of MultiWOZ 2.3 (where 20.16% of dialogues contain coreference), we apply the same idea to the CDST "coref vs. none" slot classifier. A slot that is undecided at turn $t$ stays pending, and a committed coreferred value can be re-typed or re-extracted when a later turn contradicts it. Joint Goal Accuracy is the natural metric here because a single early slot error zeroes every later turn, as the turn-weighted loss paper demonstrated.

Baselines

The naive reference point is commit-at-arrival. The same detector and resolver as our system run on the dialogue prefix, and each trigger's decision is frozen at the turn where it appears. It shares weights, encoder and training data with the proposed system, so any difference is attributable to revision alone. It is also the operating mode implicit in every pipeline reviewed above when used online.

The strongest published pipeline on these dialogue corpora is Pipeline Coreference Resolution Model for Anaphoric Identity in Dialogues. It reports CoNLL F1 of 68.27 on Light, 48.87 on AMI, 69.06 on Persuasion and 60.99 on Switchboard (sub-task 1 test sets), using full-dialogue offline processing. We re-implement it on our encoder and report it in two forms: offline, matching the published setting, and online with commit-at-arrival. For discourse deixis, the strongest reported figure is Anikina et al.'s CoNLL F1, which ranges from 35.91% to 62.79% across corpora in Anaphora Resolution in Dialogue: System Description. We run their multi-task type-plus-antecedent model as the joint-training competitor. On MultiWOZ 2.1, the reported numbers are 56.47% JGA for SUMBT+CDST and 59.50% JGA for TripPy-TS. We take TripPy-TS as the strongest competitor and also add our revision layer on top of it, so the comparison is "with versus without revision" rather than a contest between different trackers.

Two further baselines isolate the key ingredient.

  • Full recompute. The entire pipeline is re-run on the prefix at every turn and earlier decisions may change freely. This is the expensive upper bound on revision.
  • Revision without the undecided band. The detector makes hard decisions but repair operations remain available. This separates the value of deferring a decision from the value of reversing one.

All systems use the same pretrained encoder, the same training splits, and the same hyperparameter search budget (an equal number of trials on development data). We report encoder forward passes per dialogue alongside accuracy.

Beating the baselines means two things. First, a statistically significant gain in final CoNLL F1 over commit-at-arrival, concentrated in deixis and pleonastic triggers. Second, recovering at least most of the full-recompute gain at a clearly lower number of forward passes, with less decision churn. On MultiWOZ, success means a JGA gain over TripPy-TS without revision. For an LLM-rewriting baseline on CODI-CRAC we found no reported result on Lacuna, so we would run it ourselves.

Why not just…?

Why not just lower the detector threshold and let the resolver's null option reject false positives? This is the most serious threat. It is what end-to-end coreference with a dummy antecedent does implicitly, and it attacks false negatives directly. Our design actually includes it: the high-recall detector with a null option is our commit-at-arrival baseline after threshold tuning. It cannot fix a case where the resolver's null decision at turn $t$ is wrong given what turn $t+2$ reveals. The look-ahead diagnostic measures exactly how many errors fall into that category. If tuned thresholds close most of the gap between the gold-anaphor and predicted settings, we will say so and treat the revision layer as unnecessary.

Why not just train detection and resolution jointly, end to end? Joint training changes how decisions are made, not when. Anikina et al. trained type classification and antecedent selection jointly and still found selection bounded by type accuracy. End-to-end span models also need candidate pruning, which Kim et al. identify as discarding true mentions in messy dialogue. Our comparison against the multi-task model, run under the same online protocol, will show whether joint training already captures what later evidence provides.

Why not just re-run the whole pipeline every turn? Re-running is a correct way to revise, and we use it as an upper bound rather than dismiss it. It has two costs. Compute grows with dialogue length, which matters most on long AMI meetings. Unconstrained re-running also lets decisions flip back and forth, and a downstream consumer such as a state tracker has already acted on each version. We report flip counts and forward passes. If full recompute dominates our system at acceptable cost, the honest conclusion is that targeted revision buys only efficiency.

Why not just have an LLM rewrite each turn into a standalone query, or reason through the coreferences step by step? The conversational search literature summarized in Contextual Query Resolution for Conversational Search notes that a rewriter that substitutes the wrong antecedent misleads everything downstream, and that it is itself a commit-once stage. CoSafe found that prompting models to resolve coreferences before answering made them over-defensive. Hallucination-snowballing work such as MMHalSnowball shows that models conditioned on their own past outputs tend to stay consistent with them rather than correct them. We include a prompted LLM rewriter, run on the same prefixes, as a baseline on the coreference metrics.

Proposed Experiments

The decisive checks run in order, and each can end the project.

First, the look-ahead diagnostic on CODI-CRAC development data, which measures the recoverable share of detection errors by anaphor type. Second, the main comparison on the CODI-CRAC test sets. We report final CoNLL F1 (the average of MUC, B³ and CEAF), an exposed-error score that sums, over all turns, the number of gold anaphors whose current committed resolution is wrong, the flip count, and forward passes per dialogue. Systems are evaluated in the predicted setting, with the gold-anaphor setting as the ceiling. Third, the transfer test on MultiWOZ 2.1 and 2.3, measuring JGA with and without the revision layer on SUMBT+CDST and TripPy-TS. JGA's all-or-nothing scoring makes it the clearest view of whether revising a decision helps a consumer that has already acted on it. As a sanity check outside dialogue, the BNC one-anaphora pipeline gives a setting where our revision layer should have little effect. A detection-to-end-to-end drop of only 67.38 to 64.61 suggests there is little to recover there, and finding a large gain on BNC would point to a flaw in our evaluation.

The result that would most change practice is a clean separation by type: deixis and pleonastic "it" benefit from waiting, while identity coreference does not. That would tell builders of dialogue pipelines which decisions are safe to commit immediately and which should be held open for a few turns.