Referring Expressions Built Across Turns as Discontinuous Mentions

When two people share a task but see different things, they rarely name an object in one go. One says "the big grey dot," the partner asks "on the left?", and the first replies "yes, and a bit darker than the others." The description of a single object ends up spread over three turns and two speakers. Each piece is incomplete on its own ("and a bit darker" refers to nothing by itself), and the partner's turns in between decide how the pieces combine. A "no" means the next piece replaces the earlier description. A "yes" means it adds to it. Structurally, this is a discontinuous entity mention: a single mention whose words are not adjacent, split by intervening material. Discontinuous entity recognition has developed good tools for grouping such pieces, but only within one sentence of clinical text. Dialogue systems for partially observable collaboration, where each agent sees only part of the shared scene, still read references one turn at a time.

This note proposes to treat multi-turn referring expressions in collaborative dialogue as discontinuous mentions. The fragments are grouped with a graph method borrowed from discontinuous named entity recognition. The words between fragments, which here are partner turns, are read as evidence about what the partner sees rather than as noise. The claim is that grouping sub-mention fragments across turns, with each fragment interpreted in the frame of the speaker who uttered it, will resolve referents better than per-mention resolution with dialogue memory. We expect the gain to be concentrated on exactly the references that current agents get wrong.

Core Claim

Partial observability is what produces discontinuity in dialogue. Speakers build references in installments because they cannot see what their partner sees. Discontinuous-entity machinery is the right tool for this structure, with one change: the gap is informative. In clinical text, the gap in "dark and bloody stool" is words to skip. Learning to Recognize Discontiguous Entities gives such gap words their own structural parameters, but only as filler. In dialogue, the gap is a partner turn that confirms, rejects, or narrows. We will build a fragment graph whose edges are scored conditioned on that gap content and on a belief over the partner's view. Composite mentions are then decoded as maximal cliques, meaning groups of fragments that are all pairwise linked and cannot be extended further.

Relation to Prior Work

The method comes from graph-based discontinuous NER. Discontinuous Named Entity Recognition as Maximal Clique Discovery uses a segment extractor to find text fragments and an edge predictor to link fragments of the same entity, then decodes with the Bron-Kerbosch clique algorithm. A Span-Based Model for Joint Overlapped and Discontinuous Named Entity Recognition similarly classifies relations between spans. We reuse the fragment-graph-plus-clique design, but apply it across turns and speakers rather than within a sentence. The Mac paper reports that its model struggles when fragments are separated by a single word. That weakness constrains our design only mildly, because our gaps are whole turns that carry content, the opposite regime.

The motivation comes from grounded dialogue under partial observability. Reference-Centric Models for Grounded Collaborative Dialogue detects referring spans per utterance, resolves them to dots with a conditional random field (a structured model that scores joint assignments), keeps a referent memory, and names single-turn planning as a limitation. Modeling Perspective-Dependent Ambiguity in Collaborative Dialogue treats the partner's unseen view as a latent variable updated by Bayes' rule. We borrow that belief to interpret partner fragments in the partner's frame, but we use it for reading dialogue rather than for symbolic planning. The PhotoBook Dataset shows that references compress over rounds and that dialogue history helps resolution only modestly. The survey Pragmatics in Language Grounding frames partial observability as the driver of this pragmatic behavior.

The closest bridge in the file, A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition, chains LLM agents for flat NER. It models neither discontinuity nor partial observability, so it does not address this question.

Research Program

The first step is a prevalence and annotation study on OneCommon, a benchmark in which two players each see seven dots in partially overlapping views and must find one they share. We will annotate a few hundred dialogues, plus a PhotoBook subset, marking every fragment that contributes to identifying a referent, including non-standalone pieces such as "yes, darker." Fragments are grouped into composite mentions with gold referent sets. This step is decisive for the whole proposal. If fewer than roughly one in ten resolved referents involves a cross-turn fragment, the idea reduces to a curiosity, and we will report that finding.

The model encodes the dialogue with speaker tags and extracts candidate fragments. It then scores each pair of fragments $(f_i, f_j)$ for three relations: extends, replaces, or unrelated. The scorer sees the text between the two fragments and a summary of the current belief $b$ over which dots the partner can see. Maximal cliques over extends edges give composite mentions, and replaces edges prune superseded fragments. Each composite mention $M$ is resolved by combining its fragments:

$$p(r \mid M) \propto p(r)\prod_{f \in M} \sum_{v} b_{s(f)}(v)\, p(f \mid r, v).$$

Here $r$ is a candidate referent set, $s(f)$ is the speaker of fragment $f$, $v$ ranges over possible views of that speaker, and $b_{s(f)}$ is the belief over those views. Our own fragments have a point-mass belief. The resolved composites replace per-mention inputs to the existing referent memory and pragmatic generation of Reference-Centric Models for Grounded Collaborative Dialogue, so every change downstream is attributable to the reading component.

Baselines

The naive reference point is the FULL model of Reference-Centric Models for Grounded Collaborative Dialogue, which resolves each mention independently and relies on its referent memory. The strongest existing system is FULL+PRAG, which reports 83.3% corpus choice accuracy and 46.0% human-paired task success on OneCommon (the prior best was 30.6%), from that same paper. We will retrain it on the same splits with the same encoder and tuning budget. Beating it means higher referent accuracy on the annotated multi-turn subset, with no loss on single-turn mentions, and a corpus choice accuracy gain beyond seed variance. For partner modeling, the Partner Planner of Modeling Perspective-Dependent Ambiguity in Collaborative Dialogue reports 85.2% symbolic self-play success on the hardest setting. It operates on symbolic plans, so we use its belief module inside our model rather than competing with it on self-play.

Three ablations isolate the key ingredients. The first removes clique grouping and instead multiplies over all mentions in a coreference chain. The second removes the gap-content features. The third removes partner-view conditioning by treating every fragment in the listener's own frame. A prompted LLM that reads the full dialogue and outputs fragment groups is a further baseline, run with the same annotated few-shot examples.

Why not just…?

Why not just give the resolver more dialogue history? History alone helps little. PhotoBook reports 65.43 F1 with history against 64.86 without, and Fried et al. already include a referent memory. Memory stores resolved referents, but a fragment like "and darker" never becomes a resolvable mention on its own. The multi-turn subset of our evaluation tests directly whether memory already captures these cases.

Why not just resolve each mention separately and intersect the constraints through coreference? This is the most serious threat, and it is our first ablation. It will fail if its pieces are ill-posed alone ("the other one") or if a partner's "no" should delete an earlier constraint rather than intersect with it. If the ablation matches the full model, the discontinuity framing adds nothing, and we will say so.

Why not just prompt a large LLM over the whole dialogue? In the clinical domain, Comparison of Pipelines, Seq2seq Models, and LLMs for Rare Disease Information Extraction found that generative models lose to a specialized discontinuous pipeline (38.89 against 52.2 F1) because they miss strict matches. That evidence comes from a different domain, so we run the prompted baseline rather than assume it loses.

Why not just apply decentralized multi-agent readers to long clinical notes? This is the obvious intersection of the two directions, and we reject it. Benchmark discontinuous entities in CADEC and ShARe sit within sentences, and the file contains no evidence of fragments crossing window boundaries. Overlapping sliding windows would therefore likely match any learned coordination policy. In dialogue, by contrast, partial observability is what creates the discontinuity.

Proposed Experiments

The first experiment is the prevalence study, which gates the rest of the program. On the annotated OneCommon and PhotoBook subsets, we then measure composite-mention F1, requiring an exact match of the fragment set, and exact-set referent accuracy, reported separately for single-turn and multi-turn references. On the full OneCommon data, we report corpus choice accuracy and self-play success against FULL+PRAG. A small human-paired evaluation follows only if the corpus gains hold.

The decisive check is the ablation grid. The claim survives only if the full model beats the coreference-intersection ablation on multi-turn references. Separately, the partner-view conditioning must help specifically on fragments uttered by the partner. As a transfer probe, the gap-aware edge scorer will be run on CADEC. On CADEC, Mac reports an average F1 gain of 2.6 to 3.5 points over prior models but gives no absolute score in the file, and A Unified Generative Framework for Various NER Subtasks reports 70.64 F1. The probe tests whether modeling gap content, not just gap position, also helps with single-word-gap errors.