Hospitals that want to train a model to read patient timelines out of clinical notes face two obstacles at once. They cannot pool their notes, and they do not label time the same way. One corpus annotates whether an event "contains" another. Another annotates "before / after / overlap." A third mainly records how each event relates to the date the note was written. Federated learning trains one model across sites without moving raw data. In its standard form, it averages model weights or the output scores of a shared classifier, which assumes every site predicts the same set of labels. We propose that sites exchange something more basic than labels. Every temporal relation label is a shorthand for how the start and end points of two events are ordered. If each site's model predicts those orderings on shared public text, the server can merge predictions from sites with incompatible label sets. It can then repair contradictions in the merged timeline and teach the result back to every site.
In two sentences: each hospital trains locally on its own relation labels, but its model is parameterized as predicting the order of event start and end points. The hospitals then share only these point-order predictions on public, unlabeled case reports, and the server fuses them into a consistent timeline that every site learns from. This lets sites with different annotation schemes help each other. Plain weight averaging cannot do that without bolting on a separate output layer per schema, and it costs a small fraction of the bandwidth.
Core Claim
Clinical timeline reconstruction has converged on two lessons. First, events in a single patient's story are best placed relative to anchors such as admission and note creation time, and ranking their endpoints works better than classifying pairs. Learning to Temporally Order Medical Events in Clinical Text decomposed each event into start and stop points ranked against admission and reached 82.16% accuracy on clinical text, against 71.33% for a pairwise classifier. Second, enforcing transitivity across a document helps. Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference (CTRL-PG) adds a differentiable transitivity loss and a greedy global inference step on top of BERT, reaching 80.20 F1 on i2b2-2012. Removing the loss costs 1.44% and removing inference costs 1.83%. All of this work is centralized.
The federated clinical NLP work does not touch time. Federated pretraining and fine tuning of BERT using clinical notes from multiple silos federates masked-language pretraining and entity tagging. FedED federates static medical relation extraction by sending logits on a public proxy set instead of weights, reaching 75.09 F1 against FedAvg's 72.55 on 2010 i2b2/VA with a 10,000x bandwidth cut. FedED's distillation target is a per-pair label distribution over one fixed label set, so it inherits the assumption that all sites share a schema. Its targets are also independent pairs, so nothing in the merged signal respects the order of time.
Our claim is that point-order space is the right shared language for federating temporal extraction. It turns three problems the literature treats separately into one operation. Schema heterogeneity disappears because every schema's labels are deterministic sets of endpoint orderings. Global consistency becomes cheap because transitivity over points (simple less-than/equal/greater-than) is far easier to enforce than interval algebra. Bandwidth stays at FedED's logit-sharing level. What is new is not federated temporal extraction as such, which would be the obvious and shallow move of running FedAvg on a relation classifier. What is new is cross-schema federated distillation through a shared latent interval representation, with consistency repair performed on the fused teacher signal at the server rather than on any single model.
Research Program
Each event mention $e$ gets a start point $e^s$ and a stop point $e^t$. For an event pair $(a,b)$, the local model predicts, from the encoded note $x$, a three-way distribution (<, =, >) for each of the four endpoint comparisons $(a^s,b^s)$, $(a^s,b^t)$, $(a^t,b^s)$, $(a^t,b^t)$. Time expressions and the note creation time are treated as degenerate intervals, which is how anchoring enters. A configuration $c$ is one joint assignment of the four comparisons. Each relation label $r$ in a site's schema corresponds to the set $C(r)$ of configurations consistent with it; for example, BEFORE requires $a^t < b^s$, and CONTAINS requires $a^s \le b^s$ and $b^t \le a^t$. The model is trained on a site's native labels by maximizing the marginal likelihood
$$ p(r \mid x) = \sum_{c \in C(r)} \prod_{k=1}^{4} p(c_k \mid x), $$
where $c_k$ is the $k$-th comparison in configuration $c$. Comparisons a site's schema never constrains simply stay marginalized out locally. This is exactly where another site's knowledge can fill in.
Federation proceeds in rounds over a public proxy corpus. We use the event spans of the E3C clinical case corpus (labels unused), and in a scale-up, case reports from PMOA-TTS with spans proposed by the current student. Each site's model emits endpoint-comparison distributions for candidate pairs on the proxy documents and sends only these. The server fuses them with a per-site, per-comparison weighted log-linear pool. The weights come from each site's calibration on its own held-out data, reported as a scalar, so a site that never sees CONTAINS-style supervision does not dominate the start-start comparison.
The server then repairs the fused point graph by minimizing a soft transitivity penalty over points, in the Łukasiewicz form CTRL-PG uses. We choose soft repair deliberately rather than hard projection. Analysing zero-shot temporal relation extraction on clinical notes using temporal consistency found that forcing 100% consistency with integer programming on miscalibrated predictions lowered F1. That result constrains our design: repair strength is a tuned knob, and its effect is something we measure rather than assume. The repaired distributions are distilled into each site's model alongside its local loss, with a site-specific top layer in the spirit of FENDA-FL. FENDA-FL separates shared and site-specific feature extractors for heterogeneous clinical tables, and here the same split absorbs site documentation habits.
The analysis targets three questions. Does knowledge flow across schemas, for example from THYME's containment labels into i2b2's overlap predictions? Does server-side repair help or hurt under the miscalibration the zero-shot study warns about? Which site gains most? AGAT reports gains of up to 4.5% for small sites in structured-EHR federation, which motivates the hypothesis that low-resource sites gain most here too. AGAT federates hourly vitals with patient-similarity graphs and never reads text, so it informs only the expected shape of the gains.
The closest system-level work, Federated Timeline Synthesis, builds patient timelines from structured timestamps and frozen note embeddings and trains a federated generator. It never extracts when narrated events happened. Our work supplies the missing text-to-timeline step and does not compete with its generative goal.
Baselines
The naive reference point is local-only training: each site trains the same encoder with the endpoint parameterization on its own labels and never communicates. Every federated method must beat it on every site, and especially on the smallest one, or federation is not worth its overhead.
The strongest existing competitor is multi-head FedAvg. A shared clinical BERT encoder is averaged across sites by FedAvg, with a separate output layer per annotation schema kept local. This is the natural way a practitioner would federate schema-heterogeneous data, it moves full weights, and it lets the shared encoder absorb cross-site signal implicitly. Pairing it with CTRL-PG's transitivity loss inside each site gives the strongest version. The second competitor is FedED applied per schema: logit distillation on the same proxy corpus, but in each schema's own label space, with fusion only among sites that share a schema. All methods use the same encoder initialization, the same proxy documents, the same number of communication rounds, and the same hyperparameter search budget, tuned on site-held-out development sets. Beating them means higher native-metric F1 on at least two of three sites with no site worse by more than seed noise, at equal or lower bytes communicated.
Two reference numbers anchor scale. The centralized pooled upper bound on i2b2-2012 is CTRL-PG at 80.20 F1 (CTRL-PG). GRAPHTREX reports 68.81 Tempeval F1 in an end-to-end span setting (Temporal Relation Extraction in Clinical Texts: A Span-based Graph Transformer Approach). For THYME, the file reports a 1.4-point gain on CONTAINS from structured learning (Structured Learning for Temporal Relation Extraction from Clinical Records) and SVM container F1 of 0.53 (Temporal information extraction from clinical text). No absolute number for the strongest THYME system (End-to-end clinical temporal information extraction with multi-head attention) was found on Lacuna, so we will rerun it centrally as our THYME upper bound. No reported result was found on Lacuna for any federated temporal relation extraction system, so the federated baselines are ours to run.
Three ablations isolate the key ingredient. The first replaces endpoint fusion with per-schema label fusion while keeping everything else fixed; this tests whether the shared representation, not distillation itself, drives the gains. The second removes server-side repair. The third replaces soft repair with hard integer-programming projection.
Why not just…?
Why not just run multi-head FedAvg and let the shared encoder handle cross-schema transfer? This is the real threat. An encoder trained on THYME containment and i2b2 ordering may learn useful temporal features without any explicit shared output space. Our bet is that implicit sharing transfers representations but not constraints. A start-start ordering learned at one site never becomes a training signal for a comparison another site's schema leaves unconstrained. The endpoint-versus-label fusion ablation, run against multi-head FedAvg at matched rounds, settles this. If multi-head FedAvg matches us, the contribution reduces to a bandwidth saving, and we would report it that way.
Why not just prompt a large language model locally at each site? The zero-shot study (Analysing zero-shot temporal relation extraction on clinical notes using temporal consistency) found five such models trailing supervised systems by about 0.2 F1 on i2b2-2012. It also found that consistency repair did not close the gap. We include one open model as a reference row and do not expect it to compete.
Why not just map every schema to a common label set by hand and run FedED? Hand mapping loses information in both directions. CONTAINS and OVERLAP are not interchangeable, and refinements like those in Defining and Learning Refined Temporal Relations in the Clinical Narrative show that fine distinctions carry learnable signal. The endpoint representation is a lossless common denominator, which a coarse mapping is not. The per-schema FedED baseline, plus a mapped-label FedED variant, will show how much information the mapping loses.
Proposed Experiments
The main setting has three silos with native schemas: THYME colon notes, THYME brain notes, and i2b2-2012 discharge summaries. The two THYME silos share a schema but differ in specialty, which tests heterogeneity in documentation style. The i2b2 silo differs in schema. Each silo is evaluated on its own test split with its own official metric, with gold event spans first and then end-to-end, because the literature shows sharp drops once spans are predicted. A stress test adds MERLOT as a French-language fourth silo, using a multilingual encoder, to see whether point-order sharing survives a language shift. The reported 0.04 English–French gap on document-time relations (Temporal information extraction from clinical text) gives a reference for how large the language effect alone might be.
Three decisive checks will settle the proposal. The first is cross-schema transfer. We subsample the i2b2 silo to 10%, 25%, and 50% of its training documents and ask whether its F1 rises faster under endpoint fusion than under multi-head FedAvg, since that is where shared constraints should matter most. The second is consistency versus accuracy. We plot site F1 against transitivity violation rate across repair strengths, from no repair through soft repair to hard projection, and check whether the zero-shot study's negative result reappears when the inputs are fused supervised teachers rather than zero-shot outputs. The third is communication. We report bytes per round and F1 per megabyte against FedAvg, expecting FedED-like savings, because only four three-way distributions per proxy pair are sent.
We will also report a known limitation honestly. Recurring events such as chemotherapy cycles break the one-interval-per-mention assumption, as CTRL-PG and Cross-narrative Temporal Ordering of Medical Events note. We will measure error rates on those mentions separately rather than claim the representation covers them.
References
- Learning to Temporally Order Medical Events in Clinical Text
- Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference
- Federated pretraining and fine tuning of BERT using clinical notes from multiple silos
- FedED: Federated Learning via Ensemble Distillation for Medical Relation Extraction
- PMOA-TTS: Introducing the PubMed Open Access Textual Times Series Corpus
- Analysing zero-shot temporal relation extraction on clinical notes using temporal consistency
- FENDA-FL: Personalized Federated Learning on Heterogeneous Clinical Datasets
- Federated Learning For Heterogeneous Electronic Health Records Utilising Augmented Temporal Graph Attention Networks
- Federated Timeline Synthesis: Scalable and Private Methodology For Model Training and Deployment
- Temporal Relation Extraction in Clinical Texts: A Span-based Graph Transformer Approach
- Structured Learning for Temporal Relation Extraction from Clinical Records
- Temporal information extraction from clinical text
- End-to-end clinical temporal information extraction with multi-head attention
- Defining and Learning Refined Temporal Relations in the Clinical Narrative
- Cross-narrative Temporal Ordering of Medical Events