Insurance claims record three kinds of events for each patient: what they were diagnosed with, what procedures they had, and what drugs they filled. Prediction models usually merge all three into one timeline of codes. Clinically, though, these streams interact. A normal blood-sugar test means something different for a patient filling insulin than for one who is not, and a heart-failure code paired with a loop-diuretic refill tells a different story than the code alone. Neither stream carries that information alone. It exists only in the combination, which is what information theory calls synergy. We propose to measure synergy patient by patient in large claims databases, using a recent fast estimator, and to test a specific, falsifiable claim: the patients whose risk depends on synergy between the treatment stream and the diagnosis/lab stream are the same patients for whom models break when a stream is partially missing or when the model moves to a new institution.
This matters because claims are incomplete in predictable ways. Pharmacy fills paid out of pocket never appear, lab values are often missing or binned, and coding habits differ across sites. A model trained on a national database already loses more than 10% AUPRC when applied to a regional hospital (Distillation to Enhance the Portability of Risk Models Across Institutions with Large Patient Claims Database). If that loss concentrates in a small, identifiable high-synergy subpopulation, there is a cheap fix: flag those predictions at deployment or route them to a fallback. Retraining the whole model would not be needed.
Core Claim
Partial information decomposition (PID) splits what two inputs $X_1, X_2$ tell us about an outcome $Y$ into redundancy (shared by both), uniqueness (in one only), and synergy (present only jointly, as in XOR). We take $X_1$ to be the treatment stream (pharmacy dispenses and procedures) and $X_2$ to be the physiology stream (diagnosis codes and, where available, lab results). Grouping the data into two streams is deliberate. The sample-level estimator we reuse is defined only for two sources, and three-way decomposition remains unsolved (Efficient Quantification of Multimodal Interaction at Sample Level).
We make three predictions, in order of increasing risk. First, synergy in claims is concentrated in "treatment-masks-physiology" situations, where a drug changes what a code or lab value implies. Second, gains from joint over single-stream models concentrate in high-synergy patients. Third, high-synergy patients account for a disproportionate share of the accuracy lost under stream dropout and cross-site transfer.
The third prediction carries the mechanism. Analyses of artificial networks found that synergistic units have no redundant backups and fail sharply when one input is removed, while redundant units survive (Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks). The same estimator paper reported that synergy-heavy samples govern out-of-distribution behavior. Neither result has been tested on irregular longitudinal medical data. Both bear directly on the portability problem in claims.
Relation to Prior Work
Prior multimodal clinical models use the word "synergy" or align modalities, but none measures the decomposition. FlexCare fuses ICU time series, chest X-rays and notes in MIMIC-IV with covariance regularization to discourage redundancy. Its "synergy" means transfer across tasks, and it never quantifies cross-modal interaction. MEDHMP aligns notes with billing codes contrastively. That objective rewards shared information, so it amplifies redundancy rather than isolating synergy. MixEHR-Guided is the only work here run on a multi-year claims cohort (PopHR, 1.3M patients). It showed that prescriptions such as tiotropium reveal COPD patients missing the diagnosis code, which is direct evidence that the treatment stream carries information absent from diagnoses. However, its bag-of-codes topic model discards time and cannot separate unique from synergistic information. These papers motivate the gap, and all of them work on acute-care data or on unordered feature sets.
The claims-modeling line supplies our encoders. RareBERT tags each event as diagnosis, procedure or prescription with a type embedding and adds time-from-anchor embeddings. CLMBR shows that language-model pretraining on code timelines beats count features. MOTOR scales time-to-event pretraining to MarketScan. Razavian et al. model raw lab trajectories in claims. All of these treat the streams as one sequence and never ask which predictions depend on stream interactions. The portability line (Distillation to Enhance the Portability of Risk Models Across Institutions with Large Patient Claims Database, YAIB) documents cross-site loss but does not say which patients lose. One result constrains us. YAIB found that LightGBM roughly matches deep models for ICU mortality, with AUROC 0.84 to 0.87 across sites. Interaction structure may therefore be weak for some outcomes, and our design treats "little synergy" as a reportable result rather than a failure.
Research Program
We first build three predictors per outcome under one training recipe: a treatment-only encoder, a physiology-only encoder, and a joint encoder. All three are RareBERT-style transformers with type and time embeddings. The joint model uses late fusion with modality-specific learning rates, following Improving information fusion on multimodal clinical data in classification settings, so that one stream cannot dominate optimization and suppress interactions. For each patient we then compute redundancy, uniqueness and synergy with LSMI, the Lightweight Sample-wise Multimodal Interaction estimator. LSMI splits pointwise mutual information into positive and negative parts and estimates densities with a neural mixture estimator (KNIFE). On video benchmarks it cut evaluation time from 13 hours to 11 minutes (Efficient Quantification of Multimodal Interaction at Sample Level), which is what makes per-patient PID at claims scale feasible.
Before trusting the estimator on clinical data, we validate it semi-synthetically. We take real claims timelines and define planted outcomes three ways: from one stream only, from features duplicated across streams, or as an XOR of a drug-class fill and a diagnosis or lab condition. LSMI must recover the planted type per patient.
We then use the synergy maps for a deployment-time intervention. When a patient's synergy score is high and one stream looks incompletely captured (for example, no pharmacy activity for a chronic-disease patient), the model abstains or falls back to the single-stream predictor. We compare this against generic uncertainty-based abstention. As a secondary training intervention, we test synergy-weighted stream dropout against uniform stream dropout. The known risk is that dropout trades synergy for redundancy and may simply erase the signal.
Data are MERATIVE MarketScan for training. The 1.1M-patient OMOP cohort and readmission task of the distillation paper serve as the outcome definition, with VUMC as the target site. STARR-OMOP is a second target. Lab-linked analyses use the 298K-person, 18-lab claims cohort of Razavian et al.. Outcomes are readmission, 1-year heart-failure onset and COPD identification. COPD is chosen because MixEHR-G already shows a Rx–Dx information pattern for it.
Baselines
The naive reference points are outcome prevalence and a LightGBM model on hand-built summary features: counts per code family, Charlson-type comorbidity score, and lab min/max/trend. YAIB suggests this baseline can be close to deep models. If it matches the joint model, the synergy question is moot for that outcome, and we report it as such. The second baseline is a single-sequence claims model that mixes all streams in one timeline, a RareBERT/CLMBR-style transformer. This is how the field currently handles claims, and the reason to separate streams at all must be shown against it. The third is late fusion with adapted learning rates from Improving information fusion on multimodal clinical data in classification settings. On MIMIC-III mortality it reported AUC-PR 0.620 versus 0.570 for early concatenation, making it the strongest fusion recipe in the file.
For portability, the strongest existing method is Born-Again Network distillation. It reported readmission AUPRC 0.308 at VUMC, against 0.300 for local-only training, per Distillation to Enhance the Portability of Risk Models Across Institutions with Large Patient Claims Database. We rerun it on our cohort definitions rather than reuse that number. For time-to-event comparisons, MOTOR reports an average 4.6% time-dependent C-statistic gain over Cox and random survival forests (MOTOR), but no absolute value for our tasks is in the file. We will therefore fine-tune a MOTOR-style model under the same budget if weights are obtainable, and otherwise report no reported result found on Lacuna for these exact tasks.
The key ablation isolates the PID estimate itself. We replace LSMI synergy with a cheap proxy: the change in each patient's predicted risk when either stream is masked from the joint model. Abstention is compared against predictive-entropy and ensemble-disagreement abstention.
All models share identical splits, the same encoder size, and an equal hyperparameter search budget of the same number of trials per method. Results use bootstrap confidence intervals.
Beating the baselines means three things. Synergy deciles must explain a significantly larger share of the cross-site AUPRC loss than entropy or the masking proxy does. Synergy-triggered fallback must raise AUPRC on retained patients at matched coverage beyond entropy abstention. Combined with BAN, it must exceed BAN alone at VUMC.
Why not just…?
Why not just measure each stream's importance by dropping it and watching accuracy fall? Dropping a stream measures that stream's total contribution, which mixes uniqueness and synergy. The two behave differently under shift: unique information survives the loss of the other stream, while synergistic information does not. Still, the masking proxy is the most serious threat here, because it is cheap and may correlate with synergy. The ablation in the Baselines section settles it directly. If the proxy explains portability loss as well as LSMI does, we will report that the cheap proxy suffices and drop the estimator from the recommended pipeline.
Why not just feed everything into one foundation model like MOTOR and let attention find the interactions? A single-sequence model can learn interactions, and we expect it to predict as well as our joint model on in-distribution data. Our claim is not about average accuracy. It is about which predictions depend on interactions and therefore break when pharmacy capture changes, and a single-sequence model gives no such map. We test this by applying the same stream-dropout and cross-site protocol to the single-sequence baseline. If its degradation is uniform across our synergy deciles, the fragility claim fails.
Why not just use gradient-boosted trees with hand-made drug-by-diagnosis interaction features? Where clinicians already know the relevant interactions, this works and is interpretable. The evidence against it as a general solution is mixed. RareBERT reports XGBoost at about 43% AUPRC versus 80.1% on X-linked hypophosphatemia, with a drop of more than 67% under 15% random event loss compared with 1.35% for the transformer. Meanwhile YAIB finds trees competitive on ICU mortality. We include the LightGBM baseline with an added set of known drug–lab and drug–diagnosis pairs, and we report per outcome where it closes the gap.
Why not just train with uniform stream dropout so the model never relies on interactions? That would buy robustness by converting synergy into redundancy, which matches the dropout–redundancy link in Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks. The cost is the accuracy those interactions provide on complete data. We measure that trade-off explicitly as complete-data versus degraded-data AUPRC for uniform dropout, synergy-weighted dropout, and no dropout with synergy-triggered fallback. If uniform dropout dominates, the deployment intervention is unnecessary, and we will say so.
Decisive Checks
The first check is estimator validity. On the semi-synthetic XOR, duplicate and single-stream plants, LSMI must assign planted-synergy patients to its top synergy decile far above chance. If it fails on claims-shaped data, the program stops at a negative result about the estimator.
The second check is clinical structure. We stratify real high-synergy patients by whether they fill a drug class related to their outcome, for example diuretics for heart failure or bronchodilators for COPD. This tests whether synergy is concentrated in treatment-masked physiology, the first prediction.
The third check is the core test. We simulate realistic capture loss by deleting a growing share of pharmacy events, uniformly and by plan type. We also transfer MarketScan models to VUMC and STARR-OMOP. In each setting, we ask whether the top synergy decile carries a disproportionate share of the AUPRC loss compared with the masking proxy and entropy.
The fourth check is the intervention. Synergy-triggered fallback, alone and combined with BAN distillation, should improve target-site AUPRC at matched coverage.
A clean negative is also informative. If synergy is small and does not track portability loss for these outcomes, single-sequence claims models lose little by ignoring stream boundaries, and their failures under shift come from somewhere else.
References
- Distillation to Enhance the Portability of Risk Models Across Institutions with Large Patient Claims Database
- Efficient Quantification of Multimodal Interaction at Sample Level
- Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks
- FlexCare: Leveraging Cross-Task Synergy for Flexible Multimodal Healthcare Prediction
- Hierarchical Pretraining on Multimodal Electronic Health Records
- MixEHR-Guided: A guided multi-modal topic modeling approach for large-scale automatic phenotyping using the electronic health record
- RareBERT: Transformer Architecture for Rare Disease Patient Identification using Administrative Claims
- Language models are an effective representation learning technique for electronic health record data
- MOTOR: A Time-to-Event Foundation Model For Structured Medical Records
- Multi-task Prediction of Disease Onsets from Longitudinal Laboratory Tests
- Yet Another ICU Benchmark: A Flexible Multi-Center Framework for Clinical ML
- Improving information fusion on multimodal clinical data in classification settings