Phase-Indexed Raw Waveform Models for Fine-Grained Heart Sound Diagnosis

A heart sound recording is a sequence of repeated cycles: a first valve closure (S1), a systolic interval, a second closure (S2), and a diastolic interval. Many cardiac diagnoses depend on where in that cycle an abnormal sound occurs. A murmur during systole points to different defects than one during diastole, and some defects produce a murmur that spans both. Current heart-sound classifiers mostly discard this structure. The best murmur detector in the 2022 PhysioNet Challenge deliberately skipped cycle segmentation and classified mel-spectrograms (Lu et al.). The strongest fine-grained congenital heart disease model, HSDreport, also reads whole-recording spectrograms (HSDreport). Meanwhile, very accurate cycle segmenters exist (TFAN, CLSTM), but they stop at marking phases and never diagnose anything.

We propose a single network that works on the raw audio samples. It learns a soft segmentation of each cycle, borrowing the boundary-regression training used in medical image segmentation, and uses those phase probabilities to decide how to summarize the signal. Features are averaged within each phase of each beat, then across beats, then across stethoscope positions. Each level of this hierarchy matches a physiological time scale rather than a speech-derived window. The bet is narrow and testable. Phase-indexed pooling should not beat segmentation-free models at simple murmur detection, where it is expected only to match them. It should beat them on fine-grained, phase-dependent diagnoses and on recordings where loud murmurs hide the valve sounds.

Core Claim

Diagnostic granularity, meaning the ability to output a specific defect or murmur type rather than "normal versus abnormal", requires features that are tied to cardiac phase. The cheapest reliable way to obtain phase-tied features is to train segmentation and diagnosis together, so the diagnostic loss can correct phase boundaries that matter clinically. Segmentation then becomes a pooling operator inside the classifier instead of a separate pre-filter. Let $h(t)$ be the encoder feature at time $t$, $p_k(t)$ the predicted probability that time $t$ belongs to phase $k \in \{S1, \text{systole}, S2, \text{diastole}\}$, and $w_b(t)$ a soft window over beat $b$ derived from the same phase predictions. The beat-phase summary is

$$z_{b,k} = \frac{\sum_t p_k(t)\, w_b(t)\, h(t)}{\sum_t p_k(t)\, w_b(t)},$$

and the diagnostic head sees $z_{b,k}$ aggregated over beats and auscultation sites. Because $z_{b,k}$ is differentiable in $p_k$, the diagnostic loss can move phase boundaries. This is the missing link between the two-stage pipeline of Wang et al. and the segmentation-free pipeline of Lu et al..

The closest prior work plays three different roles. Wang et al. motivates the proposal. Their frozen TFAN selects the five clearest beats for a separate spectrogram classifier, which reached 0.950 murmur-present sensitivity but only 0.601 weighted accuracy. Their beats are discarded rather than phase-indexed, and segmentation errors cannot be corrected by the classifier. Lu et al. constrains the proposal: segmentation can be a single point of failure in noise, so our model must degrade gracefully when phase confidence is low. CLSTM supplies the raw-waveform segmentation recipe. It also names the failure we target: S2 is mislabeled under loud murmurs.

Research Program

The encoder ingests raw phonocardiogram samples without the Wiener and wavelet pre-filtering TFAN depends on. Its first layer is a band-pass filterbank parameterized by cutoff frequencies, in the style of SincNet (Ravanelli and Bengio), initialized to cover the low cardiac band rather than mel spacing. This follows SoundNet's finding that learned first layers find task-appropriate bandwidths on their own (Aytar et al.). Instance normalization handles stethoscope variation, as in TFAN. Dilated residual convolutions give a receptive field spanning roughly one cardiac cycle.

The temporal hierarchy has four levels. The finest level is sample-level transients in the encoder. Next come phase summaries within a beat, then beat-to-beat aggregation (attention over $z_{b,\cdot}$), then recording-to-patient aggregation across auscultation sites. This replaces the speech-calibrated windows that the wav2vec 2.0 heart-sound study itself flagged as mismatched (Exploring Wav2vec 2.0). It also follows Foucard et al., who found that combining widely separated scales beats a single compromise scale.

The segmentation branch has two heads. One is a per-sample phase classifier with TFAN's state-transition penalty. The other is an auxiliary head that regresses a Gaussian heatmap centered on each phase boundary. This is the one-dimensional analogue of HF-UNet's contour regression (He et al.), which in prostate CT cut average surface distance from 3.892 mm (U-Net) to 1.366 mm. Cross-entropy alone weights every sample equally. The boundary head instead focuses supervision on the onsets that murmurs obscure.

Training mixes data sources. Recordings with phase labels (PhysioNet 2016, MITHSDB) supervise segmentation. Diagnostic datasets (CirCor, HSDreport) supervise the diagnostic head and also receive pseudo-phase targets from an initial segmenter. Pseudo-phase loss weight is annealed so the diagnostic gradient can increasingly reshape boundaries.

A phase-confidence score is computed as the mean entropy of $p_k$ over beats. It gates a fallback to a phase-agnostic pooled feature. This directly answers Lu et al.'s single-point-of-failure argument and is evaluated as an abstention signal for the "Unknown" class.

Baselines

The naive reference is the same raw-waveform encoder with global average pooling over time and no segmentation branch. It shows what the encoder gets without phase structure.

The strongest existing method for coarse detection is the segmentation-free AudioClassifier of Lu et al.. It is a mel-spectrogram CNN with a demographic side branch and reported a weighted murmur metric of 0.780 on the PhysioNet 2022 hidden test set. The hidden set is not available for new submissions. We will therefore reimplement it and train it on the same patient-level cross-validation folds of CirCor, with the same hyperparameter search budget as our model. We will report both their published number and our reproduction, so the reader can see any reproduction gap.

For raw-waveform self-supervision, we will fine-tune wav2vec 2.0 as in Exploring Wav2vec 2.0, which reported weighted accuracy 0.80, unweighted average recall 0.70 and murmur recall 0.86 on its holdout. It is the main threat on data efficiency. We will also run it at the 13% label fraction where it was reported to hold steady.

For fine-grained diagnosis, the strongest baseline is HSDreport (Zhao et al.). It is a spectrogram ResNet50 aligned with GPT-4-extracted definitions by contrastive learning and reports 75.25% F1 and 73.38% recall over 12 categories. We will compare on its benchmark splits. Our variant will also accept the same text-alignment loss, so that any gain is attributable to phase indexing rather than to the absence of text supervision.

For segmentation, the references are TFAN (96.72% average F1; 91.31% on its hardest LEVEL-III subset, versus 78.46% for the LR-HSMM) and CLSTM on MITHSDB (96.18% F1 at 100 ms tolerance, 94.37% at 40 ms).

The decisive baseline is the two-stage pipeline of Wang et al., rebuilt with our encoder: a frozen segmenter followed by phase pooling. It isolates joint training.

Three ablations each remove one ingredient. The first drops the boundary heatmap head. The second replaces phase pooling with unconstrained learned attention pooling with matched parameters. The third replaces the Sinc filterbank with a mel-spectrogram front end.

The criteria for success are set in advance. On CirCor three-class detection, we count parity with the Lu reproduction as success, not a win. On HSDreport, the proposal succeeds only if it exceeds the reproduced HSDreport F1 with non-overlapping bootstrap intervals, and particularly on categories whose murmurs are expected to be phase-specific. On segmentation, it must improve the 40 ms-tolerance F1 on murmur-heavy recordings over a CLSTM-style model trained on the same data.

Why not just…?

Why not just use the segmentation-free spectrogram CNN, since it won the challenge? For three-class murmur detection it may well be as good as our model, and we predict roughly that. Its weakness appears where the label depends on timing. Lu et al. ranked 10th on clinical outcome prediction despite ranking 1st on murmurs, and a whole-recording spectrogram has no explicit way to tell a systolic murmur from a diastolic one. The HSDreport comparison and the phase-shuffle test below settle whether that weakness is real. If the spectrogram CNN matches us on HSDreport categories too, the proposal's main claim is wrong.

Why not just freeze an accurate segmenter like TFAN and pool on its output? This is the most serious alternative. TFAN already reaches 96.72% F1, so a frozen segmenter may be good enough. Our argument rests on the residual errors concentrating exactly in murmur-heavy recordings, where CLSTM reports S2 failures and TFAN drops to 91.31%. Those are the recordings that matter for diagnosis. The frozen-segmenter ablation measures this directly. If joint training gives no gain on hard recordings, we would simplify to the two-stage design and keep only phase pooling.

Why not just fine-tune wav2vec 2.0 with attention pooling and let it learn phase implicitly? Attention could in principle discover phase. The matched-capacity attention ablation and wav2vec fine-tuning test this. We also probe whether wav2vec attention weights correlate with labeled phases on PhysioNet 2016. If unconstrained attention matches phase pooling on fine-grained labels, the explicit structure is unnecessary. In that case the remaining contribution would be interpretability, not accuracy, and we would report it that way.

Proposed Experiments

Segmentation is evaluated on PhysioNet 2016 and MITHSDB at 40 ms and 100 ms tolerances. Results are stratified by murmur presence, using the TFAN difficulty levels where reproducible.

Coarse diagnosis is evaluated on CirCor DigiScope with patient-level folds. Metrics are weighted accuracy, unweighted average recall, and Unknown-class sensitivity, with Unknown sensitivity tested against the phase-confidence gate. The bar there is low: Bondareva et al. report 0.23 Unknown sensitivity even after adding a gating stage.

Fine-grained diagnosis is evaluated on the 12-category HSDreport benchmark. We report per-category F1 and recall, and the results are the central test.

Two checks decide the claim. The first is phase-shuffle at test time: permute the phase assignment of $z_{b,k}$ while keeping features fixed. If HSDreport accuracy for phase-specific defects drops sharply while overall CirCor detection barely changes, the model is using phase information where clinically expected. The second is data-efficient training: rerun everything at 13% of labels. If phase structure acts as a useful prior, our margin over wav2vec 2.0 should widen as labels shrink.

A negative result on either check, meaning no shuffle sensitivity or no HSDreport gain over the matched attention ablation, would show that explicit phase structure is not what fine-grained heart sound diagnosis needs.