Systems that read text and extract "who did what to whom" work well when the participants sit next to the verb. They fail badly when a participant is left unsaid and must be found earlier in the document. On RAMS, a benchmark for linking event participants across sentences, one strong model reaches about 43.4 F1 on arguments in the trigger's own sentence. It falls to roughly 15–17 F1 on arguments just one sentence away (Zhang et al.). The usual remedy for scarce labels is multi-task learning: train one shared encoder on several related objectives at once. Ordinary sentence-level semantic role labeling (SRL), the task of tagging each verb's local arguments, has abundant data and looks like the natural helper task. We argue it is the wrong helper. Its supervision almost always rewards syntactic adjacency, which is exactly the cue that vanishes when an argument is implicit. Evidence from Japanese argument analysis points the same way: syntactic dependency features help explicit arguments but add noise for omitted ("zero") ones (Omori and Komachi).
The proposal is to turn existing explicit annotation into cross-sentence supervision. In a corpus that has both gold SRL and gold coreference, such as OntoNotes, we take a predicate whose explicit argument also has a coreferent mention in a different sentence. We hide the local argument and train the model to point to that distant mention. We call this "displaced-argument linking." It reuses the predicate-role pairs that SRL already provides, but its training signal has the same distance profile as the target task. The research question is whether matching the distance profile of the auxiliary task is what makes multi-task learning help implicit arguments. Two alternative explanations are that any extra role data helps, or that the fix lies in better gradient balancing.
Core Claim
In multi-task learning for implicit argument resolution, how useful an auxiliary task is depends on whether its training signal requires cross-sentence evidence. The amount of role annotation it adds matters much less. Concretely, we make two predictions. First, jointly training on displaced-argument linking and the target task will raise F1 on arguments one or more sentences away more than jointly training on the same number of local SRL examples. Second, measured in the shared encoder, the gradients of local SRL will agree less with the cross-sentence target than displaced-argument gradients do. Here "agree" means the cosine between gradient vectors, and a cosine below zero means the two tasks pull the shared weights in opposite directions. If both predictions hold, the RAMS distance drop is partly self-inflicted by how we choose helper tasks, and it can be reduced without new annotation.
Relation to Prior Work
Three works motivate the gap. Ebner et al. introduced RAMS, argued that cross-sentence argument linking is structurally coreference, and left joint training with coreference to future work; they used only sequential transfer, from RAMS to AIDA-1, which moved 19.1 to 24.8 F1. Zhang et al. supply the head-then-span architecture we reuse. Their model first finds argument head tokens with a biaffine scorer and then expands each head to a full span. They also documented the distance collapse and attributed about 7% of errors to coreference failures. Fu et al. unify entity and implicit-role coreference on AMR graphs, a sentence-level meaning representation, and call for joint training, but their model consumes fixed pre-parsed graphs. None of the three builds an auxiliary task whose distance profile matches the target.
The closest method is the argument cloze of Cheng and Erk (2018), which blanks out an event participant and selects a filler from document entities. It learns from 40M event triples and serves as pretraining for a separate scorer. The follow-up Cheng and Erk (2019) recasts the task as reading comprehension with a pointer network. Our task differs in three ways. It operates on span-level document positions rather than head-word triples. It requires the gold filler to lie outside the predicate's sentence. It is trained simultaneously with the target rather than before it. The comparison against their sequential-pretraining regime is a central experiment rather than an aside.
Roit et al. constrain the design. They show that models fall into an "overlap trap," scoring candidates by shared tokens instead of role direction, and they fix it with role-swapped negatives, which we adopt. Their entailment-based pipeline, SRL-NLI, is also the strongest published system on ON5V and TNE. RotoGrad supplies the opposing hypothesis: that negative transfer is an optimization problem, best fixed by reweighting and rotating task gradients rather than by choosing different tasks.
Research Program
We first build the displaced-argument corpus from OntoNotes 5.0. For each predicate-argument pair, we keep it only if the argument's coreference cluster has a mention in another sentence. We then replace the local argument span with a neutral placeholder token. The target is any mention of that cluster outside the sentence, stratified by sentence distance so the distribution can be matched to RAMS. Ebner et al. found about 90% of arguments lie within five sentences, so we cap distance there. QANom nominal predicates, where 43% of arguments are already implicit (Klein et al.), provide a second source for nominalizations. Role-swapped negatives follow Roit et al.
The model is a pretrained span encoder shared across tasks, with the two-step head-and-expand decoder of Zhang et al. as the target head. The auxiliary heads are a local SRL head, a coreference head, and the displaced-argument pointer head. They share the encoder and differ only in output layers. During training, we log the per-task gradient on the shared encoder at fixed intervals. We then compute its cosine with the target gradient, separately for target examples whose gold argument is local versus non-local. This diagnostic is what lets us say why a helper task helps or hurts, not just whether it does.
Baselines
The naive reference point is single-task training of the same encoder and decoder on RAMS alone. This reproduces the setting of Zhang et al., who report 41.8 test F1 for full detection, against 40.5 for a BERT sequence-labeling baseline. We rerun it ourselves to obtain distance-stratified numbers under our own encoder.
The obvious multi-task baseline is the one implied by Ebner et al.: uniform-sum joint training on RAMS plus OntoNotes local SRL plus OntoNotes coreference. It must be matched to our method in total auxiliary examples, optimizer steps, and hyperparameter search budget. Beating it means higher F1 on non-local RAMS arguments with confidence intervals that do not overlap across five seeds, while local F1 stays within one point.
The strongest gradient-side competitor is the same uniform task mix trained with RotoGrad's magnitude and direction homogenization (RotoGrad), using a subspace rotation as that work recommends. RotoGrad has never been tested on discourse NLP, so no reported number exists for it on RAMS. If it closes the non-local gap as well as our task does, our claim weakens to "gradient conflict is the problem, and either fix works."
The strongest existing method on document-level detection is SRL-NLI (Roit et al.). It reports 29.28 F1 on ON5V cross-sentence detection, 58.34 F1 overall on TNE, and 48.34 F1 on TNE cross-sentence. The supervised NP-SpanBERT baseline it cites reaches 54.63 F1 on TNE. We evaluate on the same splits with matched encoder size.
Two ablations isolate the key ingredient. The first is a "masked-but-local" control: the same placeholder procedure, but the gold filler is allowed to be a coreferent mention in the same sentence. This holds the task format fixed and removes only the distance requirement. The second is the displaced-argument task with role-swapped negatives removed.
Why not just…?
Why not just pretrain on the cloze task and then fine-tune, as Cheng and Erk did? This is a real threat. Our corpus could help purely as extra data in either regime. Wang et al. report that intermediate supervised training can degrade general representations (Wang et al.). Yang et al. (Yang et al.) and Situ et al. (Situ et al.) found simultaneous training beats staged pipelines in their settings. None of these works tested implicit arguments, so we run sequential pretrain-then-fine-tune on the identical displaced-argument corpus. If it matches joint training, we report that the distance-matched data is the contribution and joint optimization is not.
Why not just add coreference and local SRL with a uniform loss? This is the obvious combination. Our hypothesis predicts it improves local arguments more than non-local ones, because SRL supervision is dominated by adjacent arguments. The distance-stratified comparison and the gradient-cosine diagnostic test this directly. If uniform-sum MTL lifts non-local F1 as much as our task, the core claim fails.
Why not just fix negative transfer with gradient surgery like RotoGrad? If local SRL hurts only because its gradients conflict, rotating them away should suffice without a new task. Gradient methods can, however, only remove harmful directions. They cannot create a cross-sentence signal that the auxiliary data never contained. We also run RotoGrad on top of our task mix to check whether the two fixes compound.
Why not just prompt a large language model? On ON5V, few-shot Mistral-7B reaches 12.71 F1 against SRL-NLI's 29.28 (Roit et al.). That result makes prompting a weak competitor at this model scale. We include it only as a reference row.
Proposed Experiments
The primary benchmark is RAMS, using full detection F1 broken down by sentence distance from −2 to +2. Secondary benchmarks are ON5V cross-sentence F1 and TNE overall and cross-sentence F1, with the Roit et al. numbers as references. For out-of-domain transfer, we use AIDA-1 against Ebner et al.'s 24.8 F1, and G&C, where EVENTCOMP reports 49.6 F1 (Cheng and Erk (2018)) and no reported result is found on Lacuna for the pointer reader. We also report non-core-role F1, since both Zhang et al. and Fu et al. identify rare roles as a failure point.
Three outcomes would be decisive. First, displaced-argument MTL must beat volume-matched uniform SRL-plus-coreference MTL on non-local RAMS F1 while the masked-but-local control does not. That pattern would show the gain comes from the distance requirement rather than the task format. Second, the encoder-gradient cosine between local SRL and non-local target examples must be measurably lower than that of the displaced-argument task; otherwise the mechanistic story is wrong even if accuracy improves. Third, the gain must hold under sequential pretraining. If it does, we report a data result; if it vanishes, simultaneous training is necessary. We would treat any outcome in which RotoGrad on the plain task mix matches our method as evidence against the central claim and report it as such.
References
- A Two-Step Approach for Implicit Event Argument Detection
- Multi-Task Learning for Japanese Predicate Argument Structure Analysis
- Multi-Sentence Argument Linking
- End-to-End AMR Corefencence Resolution
- Implicit Argument Prediction with Event Knowledge
- Implicit Argument Prediction as Reading Comprehension
- Explicating the Implicit: Argument Detection Beyond Sentence Boundaries
- RotoGrad: Gradient Homogenization in Multi-Task Learning
- QANom: Question-Answer driven SRL for Nominalizations
- Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling
- Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks
- Assisting Main Task Learning by Heterogeneous Auxiliary Tasks with Applications to Skin Cancer Screening