Minority Beats Are Minority Patients: Cross-Dataset Rebalancing for Image-Based ECG Classifiers

Many medical classifiers turn a heart or brain signal into a picture, such as a plotted heartbeat or a spectrogram (an image of how the signal's frequency content changes over time), and then reuse a standard image network pretrained on photographs. Rare conditions are handled by rebalancing: rare examples are repeated, reweighted, or generated synthetically. This proposal starts from a simple observation about why that combination may fail at a new hospital. In beat-level ECG databases, a rare arrhythmia is rarely spread across many patients. It usually comes from a handful of them. Duplicating or generating more images of that class therefore gives the network more copies of those few patients' electrode placement, amplitude, and baseline shape, and little new information about the arrhythmia. An image network, which is very good at picking up global shape and texture, can learn "this looks like patient 208" as a shortcut for "this is a rare beat." The shortcut holds on a split drawn from the same database and fails on a different one.

We propose to test this directly and to fix it with a cheap, targeted augmentation. The test is cross-dataset: train on MIT-BIH, evaluate on INCART, and the reverse. The fix, which we call context-transplant oversampling, builds synthetic rare-class beat images by taking each rare beat's deviation from its own patient's typical normal beat and adding that deviation onto the typical normal beats of many other patients. The rare class then covers the same range of patients as the common class, so patient identity no longer predicts the label.

Core Claim

We make two claims, and both can be falsified. The first is that image-domain oversampling of rare beats improves within-dataset scores largely by rewarding patient memorization, and that this shows up as a measurable drop in rare-class F1 when the same model is tested on an external database. A linear probe that predicts patient identity from the rare-class features should track the size of that drop. The second is that context transplant closes a meaningful part of the gap, doing better than weighted sampling and adversarially generated images at matched budgets. We treat one competitor as a serious threat: feeding the network the patient's own median normal beat alongside the target beat. If that simpler model matches the transplant, we will report the diagnosis as the contribution and recommend conditioning on patient context rather than augmenting.

The claim also predicts where the effect should disappear. In the TUAB to NMT clinical EEG transfer, labels are assigned per recording, and each recording is effectively its own patient. The rare class is therefore not confounded with a few individuals. The effect should be small there, and a large effect would count against our explanation.

Grounding in Prior Work

Two papers supply the pipeline we build on and also motivate the gap. Heartbeat Anomaly Detection using Adversarial Oversampling renders MIT-BIH beats as images and uses an InfoGAN, a generative adversarial network with an extra head that learns class-like latent codes, to synthesize rare beats. This raised F1 for ventricular escape beats (VEB) from 0.43 to 0.81 and for atrial premature contractions (APC) from 0.68 to 0.80. However, the evaluation stayed inside one internal split, and the synthetic samples were curated by eye. An Automatic Premature Ventricular Contraction Recognition System Based on Imbalanced Dataset and Pre-Trained Residual Network Using Transfer Learning on ECG Signal fine-tunes an ImageNet ResNet-18 on beat images with weighted random sampling. It reports 99.93% accuracy on MIT-BIH and 99.77% on INCART under leave-one-subject-out validation, but trains and tests within each database separately. Neither paper trains on one database and tests on the other, and that is exactly where our hypothesis makes its prediction.

Three papers constrain the design. Gradient-based Model Shortcut Detection for Time Series Classification shows that time-series classifiers latch onto isolated spurious points, with test accuracy on a poisoned class falling from 0.90 to 0.49 while training accuracy stays near 1.0. We extend the shortcut concern from injected glitches to a naturally occurring confound between patient and class. Identifying Hypertrophic or Dilated Cardiomyopathy had to pretrain on gridless ECG images to stop ResNet50 from attending to the paper grid. That is direct evidence that image networks exploit non-physiological structure in rendered signals.

A comparison of artificial intelligence–enhanced electrocardiography approaches presents a tension we take seriously. Its image-based network beat the native one-dimensional network on the external CODE cohort (C-index about 0.767 against 0.762) but lost on the disease-specific SaMi-Trop cohort (about 0.747 against 0.762). Image representations therefore do not uniformly help or hurt transfer, which is why a native 1D model is a required baseline here.

Two papers supply methods or controls. Evo-TFS argues that SMOTE-style straight-line interpolation destroys temporal and spectral structure, and evolves synthetic series under a combined time-domain and frequency-domain fitness. It is our strongest 1D oversampling competitor, but it was never tested across datasets. Amplifying Pathological Detection in EEG Signaling Pathways through Cross-Dataset Transfer Learning supplies the TUAB to NMT protocol, a finding that early layers transfer while deep layers specialize, and layer-wise learning-rate decay, which we adopt for all fine-tuning.

Research Program

The work has three parts.

Measuring confounding. For each rare class in MIT-BIH and INCART, we compute how concentrated the class is across patients, using the entropy of its per-patient counts. We then measure leakage in trained models. For each model, a linear probe is trained on penultimate-layer features of rare-class beats to predict patient identity. We also compare rare-class sensitivity on held-out beats from patients seen in training against sensitivity on patients never seen.

The transplant. Let $x_i$ be a rare-class beat image from patient $p(i)$. Let $b_p$ be patient $p$'s median normal beat, rendered the same way and aligned at the R-peak. Let $j$ be a donor patient drawn uniformly from all training patients. A synthetic beat is

$$\tilde{x} = b_j + \alpha \,(x_i - b_{p(i)}),$$

where $\alpha$ is an amplitude-rescaling factor that matches donor and source QRS amplitude. In words, the pathological deviation is kept and the patient-specific background is replaced. The operation is done on the aligned 1D waveform before plotting, so the rendering step is shared with real data.

Two automatic realism filters replace the manual curation used in the InfoGAN paper. First, a 1D classifier trained only on real beats must assign the intended class to the synthetic beat. Second, the synthetic beat's frequency-domain distance to real rare beats must fall within the range observed among real beats. The frequency-domain check is borrowed from the Evo-TFS fitness. The number of synthetic beats is matched to the roughly 600 per class that the InfoGAN paper found best.

The EEG control. On TUAB to NMT we train the same image pipeline on per-recording multitaper spectrogram images, with and without oversampling. We expect oversampling-induced leakage to be negligible there. This tests whether the mechanism is specifically patient concentration rather than image models failing to transfer in general.

Baselines

The naive reference is ImageNet-pretrained ResNet-18 fine-tuned on beat images with plain cross-entropy and no rebalancing. It sets the floor for rare-class F1 and anchors the estimate of how much each rebalancing method adds.

The strongest reported within-dataset image method is Ullah et al.'s ResNet-18 with weighted random sampling. It reports 99.77% leave-one-subject-out accuracy on INCART and 99.93% on MIT-BIH, measured within each database separately. No reported result was found on Lacuna for MIT-BIH-trained, INCART-tested performance. We will produce that number ourselves using their recipe.

The InfoGAN oversampler of Lima et al. is the strongest image-domain generative competitor, with a reported VEB F1 of 0.81 within MIT-BIH. We will retrain it on patient-disjoint training folds with the same synthetic budget and our automatic filter in place of manual curation.

In 1D, the competitors are a 1D CNN with weighted sampling and a 1D CNN with Evo-TFS oversampling. These check whether the image transform is itself part of the problem.

The key ablations share the transplant's rendering and synthetic budget and isolate what the transplant does. The first uses donor $j = p(i)$, which is augmentation without breaking identity. The second is a context-input model that stacks $b_{p(i)}$ as a second image channel.

All models share the same patient-disjoint folds, the same backbone where applicable, the same 40-configuration random-search tuning budget selected on source-dataset validation only, and the same layer-wise learning-rate decay. Beating the field means higher rare-class F1 and higher area under the precision-recall curve on the external database than every baseline. The margin must exceed the spread across five seeds, and source-dataset performance must not drop.

Why not just…?

Why not just use class-weighted loss and recalibrate the threshold at the target site? Reweighting and threshold shifts change how much each class counts, but they do not change which features the network uses. If rare-class features encode patient identity, no threshold can recover beats from unseen patients. The seen-versus-unseen patient sensitivity gap will show whether this is happening. If that gap is near zero under weighted sampling, this simpler answer wins, and we will say so.

Why not just stay in 1D? The mortality comparison shows that neither representation dominates under external shift. Our 1D Evo-TFS and weighted-sampling arms test this directly. The transplant can be applied to 1D inputs as well, so we will run it there too. If it helps both representations equally, the finding concerns rare-class confounding generally, not images specifically.

Why not just give the network the patient's own normal beat as context? This is the real threat. It removes patient identity by conditioning on it rather than by augmenting, and it needs no synthesis. It does, however, need several normal beats per patient at test time, which a short external recording may not provide, and it does not add diversity to the rare class. The context-input ablation settles this head to head. We will also run it with the context estimated from few beats.

Why not just decouple domain and class features, as in Enhancing Compositional Generalization via Compositional Feature Alignment? That method trains class and domain heads with orthogonal weights ($W_1 W_2^T = 0$) and then fine-tunes the encoder against them. Here the natural "domain" is the patient. With only a few rare-class patients, however, the patient head is fit to identities that are nearly synonymous with the class. It was also designed for missing domain-class combinations rather than frequency imbalance. We include it with a patient head as a secondary baseline and expect it to underperform when rare-class patient entropy is low.

Decisive Checks

The experiments are MIT-BIH to INCART and INCART to MIT-BIH on Lead II. They cover PVC, which is annotated in both databases, and any other beat classes whose annotation codes we verify align. Metrics are per-class F1, sensitivity at 95% specificity, and area under the precision-recall curve.

Four results would decide the claims:

  1. Image oversampling should show a larger internal-to-external drop in rare-class F1 than weighted sampling does.
  2. Across all methods and seeds, probe accuracy for patient identity should correlate with that drop.
  3. The transplant should beat the same-patient-donor ablation. If it does not, the gain comes from extra samples rather than from breaking identity.
  4. On TUAB to NMT, the spectrogram-image model should show a negligible oversampling effect. Balanced accuracy is compared against the reported 0.7370 for the pretrained temporal convolutional network.

If checks 1 and 2 fail, the core claim is wrong. If check 3 fails, the mechanism is wrong. If check 4 shows a large effect, the explanation is incomplete.