Rare diseases and rare combinations of diseases are the cases where a tired clinician most needs automated help in reading 12-lead electrocardiograms (ECGs). They are also the cases where deep models trained on skewed data do worst. Cardiologists, however, do not learn each rare diagnosis from scratch. They read it as a combination of findings they see every day: a widened QRS complex, ST-segment elevation over a particular group of leads, a leftward electrical axis. The free-text reports attached to large ECG archives already record those findings in prose. We propose to pull the findings out of the reports and train the ECG model to detect them. Each diagnosis is then predicted as a small, mostly fixed combination of finding detectors. A tail diagnosis with a handful of labelled examples only has to learn a few combination weights over well-trained detectors, not a full classifier over a high-dimensional embedding.
The text is used only during training. At test time the model sees the waveform alone, which matches emergency triage and screening, where no report exists yet. The central bet is that atomic findings are far less imbalanced than the diagnoses they make up. One finding such as "QRS ≥ 120 ms" is shared by many diagnoses, so the statistical strength of common classes can flow to rare ones through an explicit, readable intermediate layer. If the bet holds, the gain over strong baselines should grow as a diagnosis gets rarer. It should be largest on label combinations never seen together during training.
Core Claim
Free-text ECG reports are most valuable to a waveform-only diagnostic model under class imbalance and small samples in a specific way. They are not best used as a second input, and not as a holistic embedding to align against. They are best used as a source of atomic, per-finding supervision that turns each diagnosis into a structured combination. Concretely, let $f(x) \in [0,1]^K$ be the predicted probabilities of $K$ report-derived findings for ECG $x$, and let $h(x)$ be the encoder's embedding. The probability of diagnosis $c$ is
$$ p(y_c = 1 \mid x) = \sigma\left( w_c^\top f(x) + b_c + r_c^\top h(x) \right), $$
where $\sigma$ is the logistic function, $w_c$ is a sparse vector of finding weights, $b_c$ is a bias, and $r_c$ is a residual over the embedding. The weights are fit with a penalty $\lambda_w \lVert w_c \rVert_1 + \lambda_r \lVert r_c \rVert_2^2$ with $\lambda_r \gg \lambda_w$. The large residual penalty forces rare classes to explain themselves through findings first. Head classes keep enough data to use the residual when findings are insufficient. The sign pattern of $w_c$ can additionally be given a prior from a text definition of the diagnosis. This matters most when $c$ has only a few examples.
Why This Is Not Already Done
The closest work splits into three groups, and none of them uses report text this way.
The first group feeds text at inference. MVMTnet fuses translated PTB-XL notes with wavelet-transformed ECGs and reports 76.03% accuracy against 59.26% for a waveform-only Transformer. It also balances classes by discarding data, down to 12,500 records. We read that gap with caution. The PTB-XL diagnostic statements are closely tied to the report text, so a model that reads the report at test time is partly reading the answer. This paper motivates our work only in showing that the text carries diagnostic signal. It constrains the design by telling us not to require the text at inference.
The second group aligns whole reports with whole ECGs. ETP uses CLIP-style contrastive pre-training and gets 86.4% zero-shot accuracy on hypertrophy. Its zero-shot AUC on myocardial infarction is 46.4, below chance. SGERA matches embedding distributions with Stein kernels and reaches 86.44% AUC when linear probing PTB-XL superclasses with 1% labels. By the file's own account, it does not model fine-grained morphology such as local ST elevation or T-wave inversion. Both approaches compress a report into one vector. We keep the report's internal structure, meaning which findings are present and in which lead territory. That structure is exactly what ETP's MI failure suggests holistic alignment loses.
The third group handles imbalance in the embedding space. World Model Augmentation synthesizes rare-condition embeddings by applying ICD-code "action" vectors to normal ECG embeddings. It raises a linear probe's macro AUROC over 76 MIMIC-IV-ECG labels from 0.687 to 0.743, against 0.789 for full fine-tuning. It uses structured codes, not text, and it assumes a normal anchor that ignores each patient's own baseline. Our model never generates data. A patient's pre-existing findings, such as a baseline bundle branch block, enter $f(x)$ directly, so no normal anchor is needed.
Transferring Clinical Knowledge into ECGs Representation shares our "rich context in training, ECG alone at test" philosophy, but it transfers tabular labs and vitals rather than narrative findings. CheXbert supplies the method template for turning reports into labels, combining rule labelers with expert annotation. It produces flat labels for radiology, not a compositional layer that tail classes are built from.
Research Program
The work has four parts.
The first part is the finding vocabulary and extractor. We will define a vocabulary of roughly 80–120 atomic findings that cardiologists write in reports. These cover intervals, axis, rhythm features, conduction features, and ST/T changes indexed by lead territory (inferior, lateral, anterior, septal). A locally run open-weight language model will extract them from every MIMIC-IV-ECG and PTB-XL report. Running locally addresses the privacy concern raised for API models in Record2Vec. The extractor must output present, absent, or uncertain for each finding, because negation and hedging are the classic failure of keyword extraction (verbal autopsy narratives). A cascade design (medication cascade) is a fallback if a single prompt is unreliable. A cardiologist-annotated audit of about 500 reports per corpus will measure per-finding extraction precision and recall. For PTB-XL we will run extraction on the original German reports and on the English translations. This tests directly the translation drift that MVMTnet and the multilingual captioning work flag.
The second part is the finding detectors. Starting from the open ECG-FM encoder, we train multi-label finding heads on MIMIC-IV-ECG with light adapters. Early layers stay frozen to limit the forgetting risk noted in the knowledge-transfer paper. Extraction uncertainty is handled by down-weighting "uncertain" labels.
The third part is the compositional diagnosis layer, which is the equation above. For data-rich diagnoses, $w_c$ and $r_c$ are fit jointly. For a tail diagnosis with $k$ examples, we fit only the sparse $w_c$ and a heavily shrunk $r_c$. The prior on $w_c$'s sign pattern comes from extracting findings from a short textbook-style definition of the diagnosis.
The fourth part is the analysis. We will measure how the gain depends on class support. We will measure whether the learned $w_c$ agrees with clinical definitions, which serves as a readable explanation that does not depend on saliency maps. Finally, we will run failure analysis on diagnoses whose defining feature is not in the vocabulary.
Proposed Experiments
We use the benchmarks in the file and keep the protocols comparable to prior work.
On MIMIC-IV-ECG we follow the 76-label setup of World Model Augmentation, with a patient-level split. We report macro AUROC, and separately the macro AUROC over the least-frequent tertile of labels.
On PTB-XL we use the official folds and evaluate the 44 diagnostic subclasses and 19 form statements. These levels contain the real tail, unlike the 5 superclasses. We also evaluate the 1%-label superclass setting, so that SGERA's reported 86.44% can be compared directly.
For the small-sample site, we simulate a new hospital on CPSC2018. CPSC2018 has no reports, which suits our test-time-text-free claim. We draw $k \in \{1, 4, 16\}$ labelled patients per class, with total cohorts under 100 patients as described for specialized domains (Bria et al.). Each setting is repeated over 20 random draws and reported with confidence intervals.
The compositional split is the most distinctive test. On MIMIC-IV-ECG we hold out every ECG carrying certain diagnosis pairs that co-occur, for example bundle branch block with an infarction pattern. We train only on records showing each member of the pair separately, then test on the held-out pairs. This targets the combinatorial scarcity of multi-label tails named in the file.
Baselines
The naive reference point is a linear probe on frozen ECG-FM embeddings with unweighted binary cross-entropy. This is what a practitioner gets for free. On the 76-label MIMIC-IV-ECG task, the World Model paper reports 0.687 macro AUROC for its real-only linear probe. We will reproduce our own probe number rather than assume it transfers, since the encoders differ. The same probe with class-balanced loss and logit adjustment is the cheap imbalance fix, and it must be beaten on tail-tertile AUROC, not just overall.
The strongest imbalance competitor is World Model Augmentation. Its reported result is 0.743 macro AUROC on 76 MIMIC-IV-ECG labels from a linear probe trained on 721k real plus 2M synthetic embeddings, against 0.789 for a fully fine-tuned encoder. We will run our compositional layer as a probe-level method on the same split. To beat World Model Augmentation, our method must exceed 0.743 under the same frozen-encoder budget, and the gain must be larger on the rarest tertile than on head classes. If ECG-FM features differ from theirs, we re-implement their augmentation on our encoder to keep the comparison fair.
The strongest text-alignment competitor is SGERA. It reports 86.44% AUC linear probing on PTB-XL superclasses with 1% labels and 83.34% average cross-dataset AUC including CPSC2018. We will pre-train an SGERA-style aligner on the same MIMIC-IV-ECG reports and encoder as our method, with a matched GPU budget and hyperparameter search. Beating it means higher subclass/form AUROC at 1% labels and higher CPSC2018 few-shot AUROC at $k \le 4$. ETP is included as the plain contrastive variant. For full-data context only, the CNN ensemble of Strodthoff et al. reports about 0.939 macro AUC on PTB-XL diagnostic statements. We do not expect our layer to beat that on head classes, and claim only parity there.
Two ablations isolate the key ingredient. The flat-findings ablation trains the same finding heads as auxiliary multi-task targets but classifies diagnoses from $h(x)$ alone. This tests whether findings help through the composition or merely by regularizing features. The scrambled-map ablation keeps the architecture but permutes finding identities in the definition prior and drops the residual penalty asymmetry. This tests whether the clinical structure matters.
Why not just…?
Why not just use the reports as flat weak labels, CheXbert-style? This is the most serious alternative. Much of our benefit could come simply from denser supervision. The flat-findings ablation is exactly this alternative applied to ECGs. If it matches the compositional layer on the rare tertile and on the compositional split, we will report that text-as-labels suffices and that the bottleneck adds only readability. Our prediction is that the two separate most clearly at $k \le 4$ and on unseen diagnosis pairs. In those settings a flat classifier over a large embedding has too many free parameters. The compositional layer has a few dozen sparse weights over detectors trained on thousands of positives.
Why not just reweight the loss or oversample? The file reports that naive oversampling leads to memorization (Tousignant et al.). Reweighting cannot create information about a class with four examples; it only rescales gradients from those four. The balanced-loss probe is still run as a baseline. It is also combined with our layer, because the two are compatible, not competing.
Why not just align ECGs and reports contrastively and use zero-shot prompts? ETP's 46.4 zero-shot AUC on myocardial infarction shows that whole-report alignment can fail on exactly the territory-specific morphology that defines infarction. SGERA improves the overall number, but by its own limitations it does not model intra-lead morphology. Our matched-budget SGERA-style baseline will show whether a stronger aligner closes the gap on subclass and form labels. If it does, the structured layer is not needed for accuracy.
Why not just synthesize rare embeddings with a world model? Synthesis needs consecutive ECGs per patient and a normal anchor. It does nothing for diagnosis pairs never observed together, because there is no action vector for the pair. The compositional split tests this directly. Composition should generalize to the held-out pair. Augmentation can do so only if adding the two action vectors happens to work, which we will also test as a strengthened variant of their method.
Decisive Checks
The idea succeeds if three things hold together. First, per-class AUROC gain over the balanced-loss probe must rise with rarity, shown as a monotone trend when gain is plotted against log training support. Second, the method must beat both the flat-findings ablation and World Model Augmentation on held-out diagnosis pairs. Third, CPSC2018 few-shot AUROC at $k = 4$ must exceed the matched SGERA-style baseline.
The idea is falsified on any of three outcomes. It fails if extraction recall on territory-specific ST/T findings in the cardiologist audit is too low to supervise detectors. It fails if the flat-findings ablation matches the full method everywhere. It fails if the gains vanish when reports are extracted from translated rather than original German text, which would mean the structure is an artifact of one corpus's phrasing.
References
- MVMTnet: A Multi-variate Multi-modal Transformer for Multi-class Classification of Cardiac Irregularities Using ECG Waveforms and Clinical Notes
- ETP: Learning Transferable ECG Representations via ECG-Text Pre-Training
- SGERA: Stein-Guided ECG-Report Alignment for ECG Representation Learning
- World Model Augmentation for Imbalanced Multi-Label ECG Classification
- Transferring Clinical Knowledge into ECGs Representation
- Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT
- Can we generate portable representations for clinical time series data using LLMs?
- Automatically determining cause of death from verbal autopsy narratives
- A cascade of classifiers for extracting medication information from discharge summaries
- Let Your Heart Speak in its Mother Tongue: Multilingual Captioning of Cardiac Signals
- ECG-FM: an open electrocardiogram foundation model
- Towards AI-driven Next Generation Personalized Healthcare and Well-being
- Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL