Structural Domain Randomization for Joint Coreference and Entity Linking

Two tasks in reading comprehension work better together than apart. Coreference resolution groups the mentions in a text that refer to the same thing ("Barack Obama", "he", "the president"). Entity linking connects mentions to an entry in a knowledge base such as Wikidata. A pronoun is almost impossible to link by itself, but once it is grouped with a named mention it can borrow that link. Joint models that solve both tasks at once are measurably better than pipelines. These models, however, are trained on scarce human-annotated news and biomedical text, and they fall apart in new domains such as fan wikis or Reddit threads, where no such annotation exists. The obvious remedy is to synthesize training documents. Because a knowledge base already lists each entity's names, aliases, and description, it can act as a "simulator" that produces documents with perfectly known coreference chains and links. Robotics and vision have learned the hard way that models trained on simulators exploit the simulator's regularities and fail on reality. This is the synthetic-to-real gap.

This note argues that for joint coreference and entity linking, the gap that matters is mostly structural rather than stylistic. Synthetic documents get the statistics of reference wrong: how many entities appear only once, how often each entity recurs, what fraction of mentions are pronouns or nicknames, how many mentions name entities missing from the knowledge base, and how often the right entity is absent from a mention's candidate list. The proposal is to generate synthetic documents from knowledge-base "entity scripts" whose structural parameters are randomized over wide ranges. This is the text analogue of domain randomization in vision. We will test whether randomizing structure transfers to real text better than matching surface style, better than naive LLM generation, and better than zero-shot LLM prompting.

Core Claim

A joint coreference-and-linking model trained only on knowledge-base-scripted synthetic documents will transfer to real documents substantially better when the script generator randomizes reference structure than when it uses one fixed structure or rewrites text to mimic the target register. Two conditions are cleanly separated in the experiments: randomizing structure with plain style, and randomizing style with fixed structure. We predict the first closes more of the gap to a model trained on real data. We also predict a specific failure for fixed-structure training. Models trained at a fixed high coreference density will over-merge real singletons, meaning mentions that occur only once and should stay unclustered. Randomized models will not.

The claim follows from a tension in the literature. On the motivation side, Entity Linking via Explicit Mention-Mention Coreference Modeling trains a mention encoder by building trees that link mentions to each other and to their gold entity. They report that the gains (+1.63 accuracy and +7.94 Recall@64 on MedMentions) depend on coreference density and vanish in singleton-dominated corpora. Realistic Evaluation Principles for Cross-document Coreference Resolution shows that real corpora are singleton-heavy. Their evaluation uses predicted mentions, no artificial topic groupings, and separate scoring of singletons, and it drops scores by 33 F1. Anyone building a synthetic generator therefore faces a choice. Dense synthetic data gives the joint model its useful training signal but teaches the wrong prior about how often things corefer. Sparse synthetic data removes the signal. Randomization lets the model see both regimes and, we hypothesize, learn to infer density from context instead of assuming it.

What Would Be Built

The generator works in two stages. First it samples an entity script for a document, written as a vector of structural parameters $\theta$. The parameters are: a cluster-size distribution (the probability that an entity is mentioned once, twice, or many times), a NIL rate (mentions of entities deliberately removed from the knowledge base), an alias-type mix (canonical name, knowledge-base alias, nominal description, pronoun, invented informal nickname), a candidate-miss rate (aliases deliberately withheld from the alias table used for candidate retrieval), and a confusable-entity rate (entities sharing surface names with other knowledge-base entries). The script selects real knowledge-base entities that are related to each other, using knowledge-base relations, so the document has a plausible topic. Second, an LLM verbalizes the script into a document. Every planned mention is inserted as a tagged span, and the generated text is checked against the tags so that labels stay exact. This is the property vision gets from rendered ground truth. We discard any generation in which a tagged span fails to appear, or in which an extra untagged string matches a knowledge-base alias of a scripted entity.

During training $\theta$ is drawn fresh for each document from a broad distribution $p(\theta)$, and the model minimizes

$$\mathbb{E}_{\theta \sim p(\theta)}\, \mathbb{E}_{d \sim G(\theta)} \big[ \mathcal{L}_{\text{joint}}(d) \big],$$

where $G(\theta)$ is the generator conditioned on the script parameters and $\mathcal{L}_{\text{joint}}$ is the training loss of the joint model on document $d$.

For the model we reuse the spanning-tree formulation of Towards Consistent Document-level Entity Linking: Joint Models for Entity Linking and Coreference Resolution. That paper puts mentions, candidate entities, and a root node in one directed graph and trains over all spanning trees using the Matrix Tree Theorem, so every mention in a chain receives the same link. They report up to 5% F1 gains on DWIE and AIDA+, and more than 50% relative gain on hard mentions whose true entity is missing from their own candidate list. That borrowing behavior is exactly what the candidate-miss parameter is meant to train, which makes their model the natural host for the method. We do not change their objective. The contribution lies in the data distribution and in the analysis of which structural parameters drive transfer.

Research Program

The work has three parts. The first is a structural account of the gap. For each real target corpus we measure the reference statistics in $\theta$ from its gold annotation, used only for analysis. We then relate a model's transfer error to the distance between those statistics and the training distribution. We also report per-parameter sensitivity: we hold all but one parameter at a fixed value, randomize that one, and record how much of the transfer gain it accounts for. The second part is the comparison of structure against style. Each structural setting is crossed with two verbalization styles: plain encyclopedic, and target-register rewriting in which the LLM imitates Reddit or fan-wiki prose using a few unlabeled target samples. The third part tests whether structure can be estimated rather than randomized. We fit $\theta$ to unlabeled target text using predictions from the randomized model itself, then regenerate data centered on those estimates. This shows whether randomization is merely a stand-in for knowing the target, or whether breadth helps on its own.

Evaluation follows the realistic protocol of Cross-document Coreference Resolution over Predicted Mentions and Realistic Evaluation Principles for Cross-document Coreference Resolution. Mentions are predicted, singleton detection is scored separately from clustering, and no gold topic structure is used. In their setting, moving from gold to predicted mentions dropped event coreference from 81.0 to 54.4 CoNLL F1, so mention detection must be part of the test. Real World Conversational Entity Linking Requires More Than Zeroshots found mention detection to be the main bottleneck on Reddit, which reinforces the point.

Baselines

The naive reference point is zero-shot transfer of an existing linker trained on Wikipedia with no synthetic data. On the Reddit Conversational EL dataset with Fandom knowledge bases, Hoveyda et al. report a micro F1 of .076 for Flair + BLINK (Real World Conversational Entity Linking Requires More Than Zeroshots). On ConEL2-Test they report ELQ at .575 F1 zero-shot and .616 after in-domain fine-tuning. We rerun ELQ and Flair + BLINK on our splits instead of copying numbers, and treat the published figures as a sanity check. Any synthetic-only method that fails to beat ELQ zero-shot clearly on Reddit has not addressed the problem.

The strongest existing method is the spanning-tree joint model of Towards Consistent Document-level Entity Linking trained on real AIDA+ and DWIE training data. It serves as the in-domain ceiling for those two corpora. The file gives only its relative gains ("up to 5% F1", ">50% relative on hard cases"). No absolute reported result was found on Lacuna for this model on DWIE or AIDA+, so we will reproduce it and report our own numbers. Our headline metric is gap closure: the fraction of the difference between zero-shot and real-supervised performance that synthetic-only training recovers, with the same encoder (SpanBERT plus Wikipedia2Vec), training steps, and hyperparameter search budget. Beating this baseline outright is not expected on in-domain news. Success means recovering a large share of its performance there without real labels, and outperforming its cross-domain transfer to MedMentions, ZeShEL, and Reddit, where no real joint training data exists.

A pipeline baseline (mention detector, then within-document coreference, then a dual-encoder linker) trained on the same randomized synthetic data checks that joint modeling still matters under synthetic supervision. Revisiting Joint Modeling of Cross-document Entity and Event Coreference Resolution reports joint beating disjoint by 1.2 entity and 1.0 event CoNLL F1 on ECB+ with real data. We predict a larger joint advantage under synthetic data, because the candidate-miss parameter exercises precisely the borrowing that pipelines cannot perform.

The key ablation is the same joint model and generator with $\theta$ fixed at a single setting, at matched generation budget and token count. We use two settings: a "dense" one resembling naive LLM output, and one fixed at the target's true statistics as an oracle. Randomized training beating the dense setting isolates randomization as the ingredient. Randomized training approaching the oracle tells us how much target knowledge breadth can replace.

Why not just…?

Why not just prompt a large LLM to do coreference and linking directly? This is a real threat, and the file contains no reported result on Lacuna for LLM prompting on these benchmarks, so we cannot dismiss it. Prompted models must still retrieve candidates from niche knowledge bases such as Fandom wikis, where ELQ and BLINK already struggle, and they give no guarantee that a whole chain receives one consistent link. We will run a strong prompted LLM with the same retrieved candidate lists and the same realistic evaluation. If it matches the randomized joint model on Reddit and ZeShEL at comparable inference cost, the case for training rests only on cost and consistency, and we will report that plainly.

Why not just annotate a little real target data? Annotating Mentions Alone Enables Efficient Domain Adaptation for Coreference Resolution shows that labeling only mentions is 1.85 to 2 times faster than full chains and gives 7 to 14% average F1 gains in new domains (0.64 vs. 0.57 F1 on i2b2). We include this baseline at a fixed annotator-hour budget on the Reddit domains, both alone and stacked on synthetic pre-training. Mention labels supply neither cross-document clusters nor knowledge-base links. If synthetic pre-training followed by mention-only adaptation beats mention-only adaptation by itself, synthetic data adds value. If not, it does not.

Why not just fine-tune on ordinary LLM-generated text? This is the fixed-dense ablation above. The density finding of Agarwal et al. and the singleton findings of Cattan et al. predict that it over-merges. We measure this directly through singleton over-merge rate and the divergence between predicted and gold cluster-size distributions, in addition to F1.

Why not just regularize the encoder toward its pre-trained state, as in vision? Automated Synthetic-to-Real Generalization keeps a synthetic-trained network's features close to a frozen real-pretrained teacher through a KL penalty, reaching 61.1% on VisDA-17 versus 58.7% with fixed learning rates. That method protects representations but leaves the model's structural prior untouched, which is the failure we target. We add the KL penalty against frozen SpanBERT to both the fixed and the randomized conditions. If the penalty alone closes as much gap as randomization, our structural account is wrong.

Decisive Experiments

Synthetic training data is generated from Wikidata/Wikipedia for AIDA+ and DWIE, from UMLS for MedMentions, and from the relevant Fandom wikis for ZeShEL and the Reddit Conversational EL dataset. Real training splits are used only for the ceiling baseline. The metrics are linking F1 and accuracy, CoNLL F1 for coreference with singletons excluded, separate singleton-detection F1, accuracy on hard mentions (gold entity not in the mention's own candidate list), and NIL-cluster F1 on AIDA+.

Three results would decide the claim. The first is the structure-versus-style factorial: the claim holds only if randomized structure with plain style beats fixed structure with target style on Reddit and ZeShEL. The second is the over-merge diagnostic: fixed-dense models should show a markedly higher singleton over-merge rate on DWIE and Reddit than randomized models, while MedMentions, a dense domain, should show little difference. That contrast is a prediction made in advance, and seeing it would confirm the density mechanism. The third is the hard-mention test: removing the candidate-miss parameter from randomization should selectively reduce hard-mention accuracy on AIDA+ while leaving easy mentions intact, which would show that the generator teaches the borrowing behavior the joint model exists to perform. If style outweighs structure, or if the KL penalty alone matches randomization, we will report the structural hypothesis as refuted and the generator as a useful but ordinary augmentation.