Biomedical and clinical extraction tasks are usually broken into stages: first find the trigger words and entities, then link them into events or relations. Each stage passes hard decisions to the next, so a missed trigger can never be recovered downstream. The literature already shows a fix in the centralized setting: collapse the stages into one sequence tagger that predicts all the pieces at once. Real data, however, does not sit in one place, and it is rarely labeled the same way everywhere. Hospital A may have annotated entity mentions, hospital B only the links between them, and hospital C a small set of fully annotated notes. Under privacy rules these silos can only cooperate through federated learning, where clients train locally and share model updates rather than data. Today the only option in that setting is to fall back on a pipeline, training each stage on whichever silos labeled it, which brings back the error cascade that collapsing was meant to remove.
We propose to train one collapsed multi-task tagger across such silos. The tagger is topped by a conditional random field, a decoder that scores whole label sequences rather than each token alone. Each silo's partial annotation is treated as a constraint on the label sequence, and the decoder sums over every complete labeling consistent with what that silo observed. Each client then optimizes one term of the same global objective that a central server would optimize if it pooled the partially labeled data. This gives a clean definition of federated-centralized parity for structured multi-task extraction, and a test of whether shared, structurally masked transition scores help close the gap.
Core Claim
Two things should be true. First, a single tagger can learn joint event structure from silos that never hold full joint labels, provided the training signal comes from the probability of each silo's partial annotation, with unobserved label components summed out exactly by a constrained forward algorithm. Second, aggregating the decoder's transition scores across clients, with illegal transitions fixed to minus infinity rather than learned, makes the federated model track the centralized one more closely than either keeping the decoder private or dropping it.
The prediction is that the advantage over simpler approaches will be largest exactly where those approaches are weakest. Those are the regimes where few silos hold full annotations, where event types are skewed across silos, and where the number of clients grows. That last regime is the one in which Surgical Aggregation saw accuracy fall from 89.98% at 2 nodes to 74.73% at 10 nodes, against a 91.67% centralized baseline.
Position Relative to Prior Work
Two papers motivate the approach and supply its tagging schemes. BEESL collapses biomedical event extraction into per-token labels made of a trigger type, a relation role and a head offset pointing to the governing trigger. It reaches 60.22% F1 on Genia 2011. Yuan et al. do the same for emotion-cause pairs with a (indicator, relative distance) tag and reach 67.76% F1. Both assume every training sentence carries the full joint label and both train centrally. Our proposal keeps their label schemes but drops both assumptions.
The decoder side draws on Jagannatha and Yu. They show that independent per-token prediction fragments clinical entity spans, and that structured decoding fixes this (strict F 0.8210 for their approximate skip-chain model). We reuse the plain linear-chain form, because exact constrained marginalization is what our objective needs.
Three federated papers constrain the design. FedNER federates lower layers but keeps the decoder private to each client, and reaches 65.16 strict F1 on CADEC against 63.86 for single-site training. We test the opposite choice directly. Surgical Aggregation handles label heterogeneity by sharing some classification heads and keeping others private. It works at the document level with independent label heads and no structural coupling between them, which makes it the template for our strongest simple baseline. Federated Learning with Energy-Based Structured Probabilistic Inference uses a random field over clients to weight server aggregation. It uses that model only to set aggregation weights, never to decode sequences, so it is orthogonal to this proposal and could be combined with it later.
Federated Incremental NER treats a related heterogeneity problem, where new entity classes arrive over time. It addresses that problem with prototype distillation rather than marginalizing over missing labels. Finally, the claims of exact parity in vertical federated learning (Yang et al., Xie et al.) and in regularized convex models (Liu et al.) frame the question. Parity is achievable when every client optimizes a slice of one shared objective, and our construction is designed to restore exactly that property for structured multi-task labels.
Research Program
The model is a pretrained transformer encoder with a linear-chain random field over a "mention layer." That layer uses BIO tags with trigger types (in Genia, roughly a few dozen states), so it avoids the quadratic blow-up of putting the full joint BEESL label in the chain. Per-token heads predict the relation role and head offset. A fixed compatibility mask forbids structurally impossible configurations, such as a head offset that points to a non-trigger token, or an inside tag that directly follows an outside tag. Masked entries are never learned, so averaging across clients cannot reintroduce them. This directly addresses the risk, raised in the research file, that averaging transition matrices under skewed tag distributions corrupts mutually exclusive constraints.
Each client $k$ holds sentences $x$ with partial annotation $a$. The annotation defines a set $C(x,a)$ of complete label sequences consistent with what was observed. A mentions-only silo fixes the trigger tags and leaves arguments free. A links-only silo observes roles and heads, so it knows which tokens are triggers but not their types. A fully annotated silo fixes everything. The client loss is
$$\mathcal{L}_k = -\sum_{(x,a)\in D_k} \log \sum_{y \in C(x,a)} p_\theta(y \mid x),$$
where $D_k$ is client $k$'s data and $\theta$ is all model parameters. When a silo observes the argument layer, its term contributes the observed role and head likelihood directly. When it does not, that layer is summed out. The inner sum is computed exactly by running the forward algorithm on a lattice pruned to $C(x,a)$.
The global objective is $\sum_k \mathcal{L}_k$, which is precisely the centralized objective on the pooled partially labeled corpus. The natural parity reference is therefore centralized training on that pooled corpus, and we separately report the gap to centralized training with full labels. The emotion-cause task gets the same treatment. Emotion-only silos observe which clauses express emotion and sum over cause tags and distances. Pair silos observe the full tags.
Aggregation is FedAvg over all parameters, including transition scores. A FedProx-style proximal term is a secondary variant for heavy skew. Bregman random fields (Corro et al.) could later replace the sequential forward pass for speed. We leave that out of the core study because their projection-based inference does not directly give the exact constrained sums this objective relies on.
Baselines
The do-nothing reference is local-only training. Each silo trains the same architecture on its own partial labels, and a silo with no argument labels simply cannot predict events. We report the best single silo and the per-silo average.
The pipeline baseline is the realistic default. A federated trigger tagger is trained on all silos that hold mention labels and then frozen. Its hard predictions feed a federated argument classifier trained on silos that hold link labels. Both stages use the same encoder and the same number of rounds. The question is whether removing hard stage boundaries pays off under federation, as BEESL found centrally.
The strongest simple competitor is a masked multi-head tagger in the style of Surgical Aggregation. It has independent per-token softmax heads for trigger, role and head, losses masked for unobserved components, and full FedAvg. It handles partial labels with no structured decoding at all, so beating it isolates the contribution of constrained marginalization plus transitions.
A FedNER-style variant keeps our decoder but makes it private to each client, following FedNER. An ablation learns all transitions without the structural mask, to test whether averaging creates illegal paths. The upper references are centralized training on the pooled partial labels and centralized training on full labels. For the latter, the reported number is 60.22% F1 on Genia 2011 for BEESL. On the emotion-cause benchmark the corresponding figure is 67.76% F1 from Yuan et al.. We found no reported result on Lacuna for any federated or partially annotated event extraction system on Genia 2011 or the emotion-cause benchmark, so all federated comparisons are new.
Fairness rules apply to every method. All methods share the same encoder, data splits, communication rounds and local epochs, and the same hyperparameter search budget of twenty configurations each, selected on a held-out federated validation split. Beating the field means two things on Genia 2011 under the partial-annotation regimes. Event F1 must exceed the masked multi-head tagger by more than its seed-to-seed spread (five seeds). And the gap to centralized pooled training must be smaller than the masked tagger's gap at every client count from 2 to 10.
Why not just…?
Why not just run a federated pipeline, training each stage on the silos that labeled it? This is the obvious engineering answer, and it reintroduces the hard trigger decision whose errors downstream modules cannot undo. That failure is documented in knowledge base population by Yu et al. and Han et al., and centrally BEESL beat pipelines on Genia. Under federation the cascade may get worse, because the trigger stage trains on fewer, skewed silos. We measure this directly by breaking pipeline errors down into argument errors caused by missed or mistyped triggers.
Why not just mask the losses of missing tasks and use independent heads? This is the real threat. It is simple, it already copes with heterogeneous labels in Surgical Aggregation, and strong encoders may make structured decoding redundant. The case for our approach is that masking throws away cross-task information. A links-only silo tells you where triggers are, but a masked trigger head learns nothing from it. Independent heads also produce invalid joint structures, which Jagannatha and Yu found harmful. The experiments settle this by varying the fraction of fully annotated silos from 0% to 50%. If the masked tagger matches us when that fraction is 10% or less, the proposal's central claim fails, and we will report that partial-label federation needs no structured decoder.
Why not just keep the decoder private, as FedNER does? Private decoders avoid any risk of averaging away constraints. But they leave mentions-only or small silos with transition statistics estimated from their own data alone, and they cannot carry trigger-argument coupling from link silos to mention silos. With structural masking fixed, the averaging risk largely disappears. The private-decoder ablation, together with the unmasked-transition ablation, separates the benefit of sharing transitions from the harm of learning illegal ones.
Proposed Experiments
Genia 2011 is split by abstract into 2 to 10 simulated silos. Each silo is assigned an annotation profile of full, mentions-only or links-only, with labels stripped accordingly. Non-IID conditions use a Dirichlet skew over event types. The emotion-cause benchmark of Gui et al. is split the same way into emotion-only and pair silos. CADEC serves as a single-task sanity check against FedNER's reported 65.16 strict F1, which verifies that our federated setup reproduces known behavior before multi-task effects are claimed.
Metrics are official event F1 and pair F1, the parity gap to each centralized reference, the rate of structurally invalid outputs, and the pipeline error breakdown described above.
Three decisive checks define success. First, with zero fully annotated silos, the federated model should recover a substantial share of the fully labeled centralized F1, while the pipeline and masked baselines fall further behind. Second, the parity gap should grow more slowly with client count than the steep decline observed by Surgical Aggregation. Third, unmasked transition averaging should measurably raise invalid outputs under skew, while masked averaging should not. If all three hold, structured decoding is the ingredient that lets collapsed multi-task extraction survive federation. If only the first holds, the contribution narrows to partial-label marginalization, and the federated decoder story should be dropped.
References
- Federated Class-Heterogeneous Report Labeling with Surgical Aggregation
- Biomedical Event Extraction as Sequence Labeling
- Emotion-Cause Pair Extraction as Sequence Labeling Based on A Novel Tagging Scheme
- Structured prediction models for RNN based sequence labeling in clinical text
- FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning
- Federated Learning with Energy-Based Structured Probabilistic Inference
- Federated Incremental Named Entity Recognition
- Federated Machine Learning: Concept and Applications
- An Efficient Learning Framework for Federated XGBoost Using Secret Sharing and Distributed Optimization
- Efficient and Robust Regularized Federated Recommendation
- Bregman Conditional Random Fields: Sequence Labeling with Parallelizable Inference Algorithms
- The University of Sheffield System at TAC KBP 2010
- ISCAS_Sogou at TAC-KBP 2017