FedCoRe: Target-Adaptive Completion for Missing Modalities in Healthcare Federated Learning

Holger R. Roth, Ziyue Xu, Peter CnuddearXiv 2026 · arXiv

TL;DR

Multimodal federated learning models often degrade when deployment sites lack certain data modalities, such as ECGs or chest X-rays. FedCoRe addresses this by learning lightweight, feature- or logit-space corrections from sites that possess complete data, using a validation gate to safely apply these corrections at deployment. This approach avoids the risks and computational costs of generating synthetic clinical images or waveforms.

What Problem Does This Paper Address?

In medical AI, combining multiple data modalities—like Electronic Health Records (EHR), chest radiographs (CXR), and electrocardiograms (ECG)—significantly improves predictive performance. Standard multimodal Federated Learning (FL) typically assumes a uniform setup where every participating hospital collects every modality.

In reality, clinical environments are highly heterogeneous. A specialized medical center might collect all three modalities, while a smaller community clinic might lack advanced imaging or waveform data.

Fig. 1. Complete-modality evaluation contrasts with federated deployment, where sites and patients may have different subsets of EHR, CXR, and ECG.

When a model trained on complete data is deployed at a site missing a modality, its performance often plummets. Prior solutions attempted to synthesize the missing raw data (e.g., generating a fake X-ray image), but this is computationally expensive and clinically risky, as generative models can introduce hallucinations.

Methods

The authors introduce FedCoRe (Federated Cross-Modal Representation Completion). Instead of reconstructing raw data, FedCoRe learns task-aware residual corrections directly inside the model's internal representation or classifier-logit space.

The framework operates via three main steps:

  1. Paired Supervision: Clients that do have access to the modality in question (e.g., ECG) evaluate the same patient example twice: once with the modality present ($h^{+t}$) and once with it removed ($h^{-t}$). This creates a local paired view that teaches the completion module how to fill the gap.
  2. Supervision-Based Aggregation: During the FL round, the central server only aggregates completion module updates from clients that actually have the valid pairs to supervise it. Clients lacking the modality contribute zero weight, preventing them from diluting the global operator.
  3. Validation-Gated Deployment: Before deployment, a validation step evaluates whether applying the correction actually improves performance. If it does, the correction is applied; if it doesn't, a validation gate safely retains the unchanged baseline prediction ($\alpha = 0$).
The FedCoRe workflow consisting of paired training, selective aggregation, and validation gating.

Datasets

The authors constructed a multimodal benchmark using public datasets from PhysioNet:

  • MIMIC-IV (Structured EHR context)
  • MIMIC-CXR / MIMIC-CXR-JPG (Chest radiographs)
  • MIMIC-IV-ECG (Diagnostic ECG waveforms)

The target task is predicting binary respiratory deterioration within 48 hours (defined as future invasive ventilation, ICU transfer, or all-cause in-hospital death). The final cohort contains 13,914 encounters from 11,331 patients, with a positive class prevalence of 2.2%.

Key Insights and Ideas

  • Correction over Reconstruction: FedCoRe demonstrates that literal modality reconstruction isn't necessary for task performance. For missing ECGs, the validation-selected completion module actually outperformed the reference model that had access to the real ECGs (+0.1455 AUROC improvement). This indicates the module acts as a task-specific classifier-logit optimizer rather than a simple data recreator.
  • The Necessity of Safety Gating: While missing ECG signals were highly recoverable, missing CXR signals proved more volatile. In deployment cases completely lacking CXR data, the validation gate successfully recognized when corrections might be harmful and chose to retain the uncompleted baseline instead.

Key Results

  • ECG Recovery: Intentionally hiding the ECG modality reduced the baseline model's AUROC by 0.085. Implementing paired-example FedAvg restored 49.0% of this lost performance (+0.0415 AUROC).
  • CXR Recovery: In controlled tests where CXR was hidden, the effect-aware completion module successfully recovered 52.8% of the performance gap (+0.0870 AUROC on average across seeds).

Limitations and Future Work

  • Simulated Heterogeneity: The clients were simulated by masking a single centralized cohort (MIMIC) rather than using true multi-institutional data, meaning natural institutional covariate shifts were not fully captured.
  • Class Imbalance: The low prevalence of the positive outcome (2.2% across ~14k encounters) led to instability in secondary metrics like AUPRC.
  • Future Directions: The authors note that validating FedCoRe against true external multi-site clinical deployments remains an important next step.

Summary updated

Abstract

Federated multimodal models often assume every site has every modality, although hospitals differ in access to EHRs, chest radiographs, and ECGs. We study this setting on a MIMIC-derived respiratory deterioration task with simulated FL clients and introduce FedCoRe (Federated Cross-Modal Representation Completion). FedCoRe learns representation- or logit-space corrections rather than generating synthetic ECGs or CXR images. When a client observes a modality that may be missing at deployment, it evaluates the same example with and without that modality to obtain paired supervision. Only clients with such pairs update the completion module, and validation may retain the unchanged prediction. We freeze the trained multimodal predictor during evaluation so that measured differences come only from completion. Hiding ECG reduced AUROC by about 0.085; paired-example FedAvg restored 0.0415 AUROC, or 49.0% of the lost performance. We therefore report two distinct effects: paired-example FedAvg partially recovers the missing-ECG gap, while validation-selected completion is a task-specific classifier-logit correction rather than literal ECG recovery. For CXR, effect-aware completion recovers 52.8% of the loss in a controlled test where CXR is hidden. Paired-example FedAvg transfers part of this effect, but validation keeps the no-completion baseline for deployment cases whose inputs lack CXR. Thus, FedCoRe should be read as a validation-gated completion/correction framework: it can recover missing-modality signal in supported settings, but it should be deployed only when paired examples and validation evidence support that modality.
BibTeX
@misc{roth2026fedcore,
  author = {Holger R. Roth and Ziyue Xu and Peter Cnudde},
  title = {{FedCoRe: Target-Adaptive Completion for Missing Modalities in Healthcare Federated Learning}},
  year = {2026},
  eprint = {2608.18311},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.18311},
}
Figure