Closing the ECG Transformer's Accuracy Gap with Countable, Local Concepts

A Transformer trained on raw 12-lead electrocardiograms (ECGs) diagnoses heart conditions better than a simple model built on the measurements cardiologists already use, such as heart rate, QRS width, PR and QT intervals, and ST-segment shift. Nobody can say what the extra accuracy consists of. This proposal treats that gap as something to measure. We build a Transformer whose only route to a diagnosis is a transparent additive score over two kinds of concepts. The first kind is the standard clinical measurements, which the network must predict per lead and per beat. The second kind is a small, growing set of "residual" concepts. Each residual concept is a detector for a local pattern in one lead, and a new training penalty forbids it from smuggling in information from the rest of the recording. We then add residual concepts one at a time and record how much of the black-box advantage each one recovers.

The central question is how many auditable, localized, lead-specific patterns beyond the clinical vocabulary are needed to match an unconstrained ECG Transformer. The answer defines an accuracy-interpretability curve for this domain in concrete units. The second contribution is the locality penalty, which makes each concept's score depend only on a bounded stretch of signal around the time it fires. It targets "concept leakage," the known failure in which concept channels quietly encode task information.

Where This Sits

Three strands of prior work motivate the proposal, and each stops short of it. Enforcing Interpretability in Time Series Transformers: A Concept Bottleneck Framework aligns chosen Transformer layers with fixed concepts (an autoregressive surrogate and hour-of-day) and adds an unconstrained "free component" to keep forecasts accurate. That free component is the opaque part, and its size goes unmeasured. We make the residual itself structured and count it. A Wide and Deep Transformer Neural Network for 12-Lead ECG Classification won the PhysioNet/CinC 2020 challenge by fusing 22 handcrafted features with an opaque Transformer embedding. Its deep path is exactly the unexplained gap we want to decompose. ShapeFormer learns discriminative subsequences inside a Transformer but concatenates them with generic convolutional features before classification, which leaves a backdoor. Our readout has no such path.

Two papers supply methods we reuse. From Inherently Interpretable Time Series Classification via Multiple Instance Learning (MILLET) we take the idea of scoring time points and pooling them so that the explanation is exact in one forward pass. MILLET is univariate and says where evidence lies but not what it is; we attach both a lead and a named or retrievable pattern to each score. From concept bottleneck models, which route predictions through human-meaningful intermediate variables that can be edited at test time (Concept Bottleneck Models), we take the two-stage structure and the intervention test.

Two papers constrain the design. Learning Optimal Summaries of Clinical Time-series with Concept Bottleneck Models shows that static window summaries can match black boxes on coarse ICU prediction. ECG results point the other way: the Wide and Deep paper's score rose from 0.496 to 0.533 when the window grew from 10 to 15 seconds. Our concepts are therefore time-resolved rather than static. Concepts in Motion forbids all cross-concept attention in video and pays an accuracy cost on SSv2. We deliberately allow the Transformer full cross-lead, cross-time attention inside a bounded neighborhood and constrain only what each concept score may depend on. The apparent contradiction that bottlenecking sometimes improves accuracy (Traffic MSE 0.613 to 0.393 in the van Sprang et al. work) is a reason to measure the trade-off rather than assume its sign.

Core Mechanism

A 1D-convolutional front end followed by a Transformer encoder, matching the Wide and Deep deep path, reads the full 12-lead recording. From its token states, two kinds of heads produce concept tracks.

The clinical heads regress per-lead, per-beat values of standard measurements. Their training targets come free from an automatic ECG delineation algorithm, so no human annotation is needed. The residual heads each output a match score over time for one lead.

The diagnosis for class $y$ is computed only as

$$s_y = \sum_{k} w_{yk}\,\text{pool}_t\, c_k(t) + \sum_{j=1}^{J} v_{yj}\,\max_t r_j(t),$$

where $c_k(t)$ is clinical concept $k$ at time $t$, $r_j(t)$ is residual concept $j$'s match score, $w$ and $v$ are sparse linear weights, and $J$ is the number of residual concepts. There is no other path from the encoder to the output.

Because every token in a Transformer can see the whole recording, $r_j(t)$ could in principle encode global diagnostic information, which would amount to leakage by another name. The locality penalty addresses this. Let $\tilde{x}^{(t,\delta)}$ be the input with everything farther than $\delta$ seconds from $t$ replaced by the corresponding segment of a different patient's recording. We penalize

$$L_{\text{loc}} = \mathbb{E}_{x,t,j}\,\big| r_j(t; x) - r_j(t; \tilde{x}^{(t,\delta)}) \big|.$$

This keeps each residual score a function of a bounded neighborhood. With $\delta$ of a few seconds, the neighborhood still covers several beats, so rhythm-level patterns remain expressible. The same penalty, at a larger $\delta$, is applied to the clinical heads.

Residual concepts are added greedily, one per round with retraining. For each concept we record the diagnostic gain and three audit artifacts: its lead, its typical temporal extent, and its top-activating training segments. A cardiologist can then name the pattern or reject it.

Research Program

We will build the model above and sweep $\delta$ and $J$ to trace, for each diagnosis group, how accuracy grows as residual concepts are added. The three reference levels are the clinical-concepts-only model at $J=0$, the curve as $J$ increases, and the unconstrained black box.

Three analyses turn the curve into evidence.

The first is a leakage test. For the clinical concepts, we train the same readout on delineation-algorithm values instead of predicted values. If predicted concepts beat their own ground truth by a wide margin, information is leaking through them.

The second is an intervention test. On errors where a predicted clinical concept disagrees with the algorithm, we overwrite it with the algorithmic value and record how often the diagnosis becomes correct.

The third is a locality audit. On held-out data, we measure how much residual scores change under context swaps for models trained with and without $L_{\text{loc}}$.

A small blinded review by cardiologists will rate whether each residual concept's retrieved exemplars show one recognizable pattern.

Baselines

The naive reference point is a gradient-boosted tree and a sparse logistic regression on the delineation-derived clinical features alone, the analogue of the Wide and Deep wide path. It fixes the $J=0$ floor. It will be tuned with the same hyperparameter-search budget as every other model. We found no reported result on Lacuna for a wide-path-only score on PhysioNet 2020, so this number will be produced by us.

The strongest existing competitor is the Wide and Deep Transformer of Natarajan et al.. It reports a PhysioNet/CinC 2020 challenge metric of 0.533, both on the official test set and in 10-fold cross-validation with 15-second windows. We will reimplement it with 15-second windows on the same folds, backbone size, and training epochs as our model. Our target is to come within 0.01 challenge-metric points of its cross-validated score with $J$ at or below roughly 20 residual concepts.

For interpretable competitors, we will adapt ShapeFormer to 12-lead ECG in two forms, with and without its generic path, to show what that backdoor contributes. We will also run a MILLET-style conjunctive-pooling Transformer and a van Sprang–style bottleneck whose free component is a dense vector. That last baseline has a readout identical to ours, except the residual is unstructured. Beating it means matching its accuracy while passing the locality and leakage tests it is expected to fail.

The key ablations are $L_{\text{loc}}$ removed, which isolates the locality penalty, and residual concepts removed, which is the $J=0$ neural model and isolates whether the network's clinical measurements already beat the algorithm's.

On secondary benchmarks, MDOT from Bridging clinical knowledge and AI reports 99.03% accuracy and 96.38% F1 on Chapman-Shaoxing. On CinC 2017, PahNet from Prototype Learning for Medical Time Series Classification via Human-Machine Collaboration reports 0.8414 accuracy and 0.8235 macro-F1 on the three-class task. We compare against these published numbers where our splits match, and otherwise treat them as context.

Why not just…?

Why not just explain the Wide and Deep model with SHAP? Post-hoc attributions would still say nothing about what the deep path computes. The MILLET paper reports that perturbation methods are over 800 times slower than single-pass explanations and lose to them on AOPCR on a benchmark with known ground truth. We will still run SHAP on our reimplemented Wide and Deep and compare its deletion curves with our concept scores on the same records.

Why not just engineer more clinical features until the simple model catches up? This is a real threat. If it works, the honest conclusion is that the Transformer's advantage is mostly unexploited clinical measurement rather than new patterns. Our design detects this outcome directly: the $J=0$ neural model and the feature-only baseline show how much of the gap the clinical vocabulary already closes. We will also give the feature baseline an expanded set of standard ECG measurements before claiming any residual concept is needed.

Why not just use an unconstrained free component, as van Sprang et al. do? It probably matches accuracy. Its cost is that the opaque share of the prediction is neither bounded nor inspectable, which is exactly the leakage failure Direction 2 flags. The dense-residual baseline and the context-swap audit will quantify how non-local that free component is.

Why not just cluster black-box activations afterwards to find concepts? Clusters found after training are not guaranteed to be what the model uses. Our intervention test, which edits a concept and checks whether the diagnosis changes accordingly, can be applied to post-hoc clusters too, and we expect them to fail it more often.

Decisive Checks

The proposal succeeds if three things hold at once on PhysioNet/CinC 2020 cross-validation. The accuracy-versus-$J$ curve must reach the reimplemented Wide and Deep score with a small $J$. With $L_{\text{loc}}$ on, residual scores must stay stable under context swaps. And clinical-concept interventions must correct a measurable fraction of errors.

It fails informatively in two ways. If the curve plateaus well below the black box even at large $J$, local patterns cannot account for the Transformer's advantage and global interactions are involved. If the feature-only baseline already matches, the gap was measurement quality all along. Either result answers the question the field currently leaves open.