Peer-to-Peer Coreference: Letting Isolated Readers Fix Each Other's Entity Files

Picture several text-processing nodes, each reading its own private stream of news or reports: a regional newsroom, a hospital's notes, a company's internal feed. Each node keeps an "entity file" for every person, organization, or event it has seen, meaning a running summary of who or what that thing is. Often a node sees only vague mentions such as "the company" or "the suspect" and cannot tell which real entity they refer to. A neighboring node may already have the distinguishing detail. Distributed state estimation is the control-engineering field that studies how sensors sharing only small messages with nearby peers can jointly reconstruct a state that no single sensor can observe. We propose to treat cross-document coreference the same way. Coreference resolution is the task of deciding which mentions refer to the same entity. Here each entity file becomes an uncertain belief, local mentions act as noisy measurements, and nodes reach agreement by exchanging compact file summaries instead of raw text.

The new element is not only averaging embeddings across machines. Peer agreement feeds back into each node's own coreference decisions, so a node can reverse an early wrong link once its neighbors make an entity file more certain. Merges are committed only after evidence has been stable for several rounds. Current streaming coreference systems are centralized and greedy. Existing distributed coreference systems are batch jobs that pool all data. Neither works when text cannot leave its node, and neither can undo an early mistake.

Core Claim

Our claim is that a consensus+innovations estimator over uncertainty-weighted entity files can recover most of the accuracy of a centralized cross-document coreference system on a sparse peer network that never moves raw documents. "Consensus+innovations" means each node blends new local evidence (the innovations) with its neighbors' current estimates (the consensus). We expect the largest gains over local-only processing on mentions that are ambiguous at their own node but resolvable elsewhere in the network. Two mechanisms carry the claim. The first is precision weighting: each file and each mention carries an uncertainty, so a vague pronoun moves a file far less than a full name does. The second is reversible, delayed commitment: a node links two files only after the evidence for the link has persisted for $k$ rounds, and it can split them again if the evidence later reverses.

Research Program

The method borrows from three places. From Consensus+Innovations Distributed Kalman Filter With Optimized Gains, which runs local measurement updates and neighbor averaging on one shared clock and exchanges noise-reduced "pseudo-states" rather than raw readings, we take the update form and the message format. From Recursive Distributed Detection for Composite Hypothesis Testing: Algorithms and Asymptotics, which shows that nodes with rank-deficient local views can recover the full parameter under global observability, and which delays decisions by $k$ consensus rounds to protect long-run error rates, we take both the observability framing and the delayed-decision device. From Sequential Cross-Document Coreference Resolution, which assigns streaming mentions to cluster prototypes in $O(cm)$ time but irreversibly, we take the local streaming backbone. Our method differs from it in adding uncertainty to the prototypes, peer correction, and reversibility.

Each node first runs within-document coreference with a SpanBERT encoder (SpanBERT: Improving Pre-training by Representing and Predicting Spans, which pretrains span representations and reports 79.6 F1 on OntoNotes) and coarse-to-fine scoring (Higher-Order Coreference Resolution with Coarse-to-Fine Inference). Each mention chain yields an embedding $z_m$ and a diagonal noise variance $R_m$. The variance comes from a small head trained on ECB+ training data to predict how far a mention's embedding falls from its gold cluster centroid, so pronouns and generic nominals receive large variance and names receive small variance. Node $i$ keeps, for each file $e$, a mean $x_{i,e}$ and a diagonal covariance $P_{i,e}$. When a mention $m$ is assigned to file $e$, and neighbor files $f$ have been matched to $e$ with confidence $w_{ij}$, the per-dimension update is

$$x_{i,e} \leftarrow x_{i,e} + K_{m}\,(z_m - x_{i,e}) + \beta \sum_{j \in N(i)} w_{ij}\, \Gamma_{ij}\,(x_{j,f} - x_{i,e}).$$

Here $K_m = P_{i,e}(P_{i,e}+R_m)^{-1}$ is the Kalman gain for this mention, $N(i)$ is the set of graph neighbors of node $i$, $\Gamma_{ij}$ is a diagonal weight that favors whichever file is more certain on each dimension, and $\beta$ is a single consensus step size. Between updates the files follow a random walk with process noise $Q$, which lets an entity's description drift; for example, a "person of interest" can later become "the defendant." The match confidence $w_{ij}$ comes from a covariance-aware distance between files at neighboring nodes. A link becomes committed only after this distance stays below a threshold for $k$ consecutive rounds, and it is dissolved if the distance rises above a second, higher threshold.

The feedback step is what separates this from distributed averaging. After each consensus round, a node re-scores its recent local mentions against its updated files. A vague "the firm" that was greedily attached to the wrong local file can then move to the right one.

We will not claim the optimality or tracking-capacity theorems of the linear-Gaussian setting. Embedding spaces are not linear time-invariant systems, and evidence shared among peers can be counted twice. We will instead measure how calibrated the file covariances are, and we will compare plain consensus with a more conservative fusion rule to see whether double counting hurts. One theory-inspired question is cheap to test: does the trade-off in Sahu et al., where a longer delay $k$ lowers final error but slows early tracking, also appear in coreference? We will sweep $k$ to find out.

Baselines

The naive reference is local-only processing. Each node runs the same SpanBERT-based streaming clusterer on its own documents and never communicates, and its clusters are scored as if they were global. This is the floor that any communication scheme must beat.

The strongest existing method is centralized sequential cross-document coreference (Sequential Cross-Document Coreference Resolution), run on all documents pooled at one processor. That paper reports 75.3 CoNLL F1 for entity coreference and 80.8 CoNLL F1 for event coreference on ECB+. We will reimplement it with the same SpanBERT encoder and the same document arrival order, give it the same hyperparameter search budget as our method, and treat it as the ceiling. For the large-scale setting, the centralized reference is the hierarchical distributed sampler of Large-Scale Cross-Document Coreference Using Distributed Inference and Hierarchical Models. It runs Markov-chain Monte Carlo over pooled data on a master-worker cluster and reports 73.7 F1 on the Wikipedia Link Corpus, against 57.3 F1 for string matching. We will cite those numbers as pooled reference points rather than rerun the sampler at full scale. We count our method as successful if it closes at least two-thirds of the gap between local-only and centralized processing, with every node agreeing on the committed links, while sending far less data than shipping all mention embeddings.

The most dangerous competitor is one-shot centroid merging, essentially federated clustering. Each node clusters locally, sends all of its centroids once to a coordinator, and the coordinator merges them with agglomerative clustering, using the same budget and encoder as our method. A peer-to-peer variant floods centroids to every node instead of using a coordinator.

Three ablations isolate the key ingredients. The first removes uncertainty: unweighted gossip averaging, with $K$ and $\Gamma$ replaced by constants. The second removes the feedback: consensus still updates the files, but local mentions are never re-scored. The third sets $k=0$, so links are committed immediately and irreversibly. The second ablation is the decisive one. If it matches the full method, our central claim is false.

Why not just…?

Why not just centralize the documents and run the best centralized system? When centralization is allowed, that is the right choice, and it is our ceiling, not our competitor. The proposal targets settings where text cannot leave its source because of privacy, ownership, or bandwidth. The Kalman-filter literature (Consensus+Innovations Distributed Kalman Filter With Optimized Gains) is built for exactly this case. We will report the accuracy cost of decentralization honestly, measured against the pooled reimplementation of Allaway et al.

Why not just cluster locally and merge centroids once? This is a real threat. With strong encoders, one merge might capture most of the benefit. Two things should limit it. A single merge cannot repair local mistakes made before the merge, which is the error accumulation that Sequential Cross-Document Coreference Resolution itself acknowledges. A coordinator is also exactly what the target settings forbid. The experiment that settles the question compares accuracy on ambiguous mentions specifically, under equal message budgets. If one-shot merging comes within one F1 point of our method there, we will reframe the work as an analysis of when iterative consensus is unnecessary.

Why not just share entity names and match strings across nodes? This is cheap and transparent. However, string matching tops out at 57.3 F1 on the Wikipedia Link Corpus in Large-Scale Cross-Document Coreference Using Distributed Inference and Hierarchical Models, and it says nothing about mentions that have no name. Those unnamed mentions are the very cases that local ambiguity creates. We will include name-sharing as a baseline.

Why not just use a long-context transformer over all the documents? That requires putting all the text in one place, which the setting rules out. Separately, End-to-end Neural Coreference Resolution documents attention-based models confusing related roles with identical referents, so more context alone does not supply identity tracking.

Proposed Experiments

We will build a distributed version of ECB+ by spreading each topic's documents across 5 to 20 simulated nodes connected by ring, Erdős–Rényi, and star graphs. We will control local observability by assigning documents so that some nodes receive only articles in which an entity appears as a pronoun or generic nominal, while another node holds an article that names it. Results will be scored with the CoNLL metric (the average of MUC, B³, and CEAF F1) on global clusters, separately for entities and events. We will break results down by whether a mention is resolvable locally or only through the network, and we will also report bytes sent, rounds until links stop changing, and the fraction of node pairs that disagree on committed links.

For scale, we will partition the Wikipedia Link Corpus across 50 nodes by source site. For a long-tail stress test, we will distribute SemEval-2018 Task 5 documents and measure incident counting. That task's best team reached only 26.38% event-counting accuracy (SemEval-2018 Task 5: Counting Events and Participants in the Long Tail). We take that figure as evidence of difficulty, not as a directly comparable baseline, because our setting differs.

The decisive checks are three. The full method should beat the no-feedback ablation on network-resolvable mentions. It should beat one-shot centroid merging at an equal message budget. Sweeping the delay $k$ should trade early accuracy for lower final error. Finally, to test whether the process noise $Q$ behaves like a tracking-capacity knob, we will inject controlled drift into entity descriptions in time-ordered SemEval incident streams and look for the drift rate beyond which accuracy collapses.