Heart sound recordings are cheap to collect and expensive to label. A digital stethoscope can produce thousands of recordings, but each needs a cardiologist's judgment, and most of them come from healthy people. The obvious move is self-training, also called pseudo-labeling. A classifier trained on the few labeled recordings guesses labels for the unlabeled ones. Its confident guesses are added to the training set, and the cycle repeats. In medicine this backfires in a predictable way. The model is most confident on easy normal recordings, so the pool of guessed labels fills with normals. The model then becomes more sure that ambiguous murmurs are normal, and the rare abnormal class, the one that matters clinically, is the one that erodes.
We propose to break that loop with a second classifier that cannot learn the first one's mistakes. It is built on fixed, hand-designed acoustic features: wavelet filter banks plus heart-cycle timing statistics, with no learned filters. A learned 1D convolutional network on raw audio and this frozen-feature model must agree before a recording receives a pseudo-label. Abnormal pseudo-labels are admitted by an explicit quota tied to an estimate of how common abnormal recordings are in the unlabeled pool, not by a confidence threshold. The claim is that the frozen view plays the role an external language model plays in speech-recognition self-training: an independent prior that does not co-adapt with the student. It can stop confirmation bias in a domain that has no grammar to lean on.
Core Claim
Iterative Pseudo-Labeling for Speech Recognition repeatedly fine-tunes a speech model on its own decoded transcripts. It uses an external language model during decoding to correct acoustic errors. slimIPL removes that language model but needs two stabilizers to avoid divergence:
- a slowly refreshed cache of past pseudo-labels, which acts as a temporal ensemble;
- a dropout and augmentation schedule that starts heavy and decays.
Both methods were validated on balanced read English speech. Heart sounds offer neither a language model nor balance. FocalMatch shows that confidence-thresholded pseudo-labeling produces a class-skewed pseudo-label pool even when the unlabeled data are balanced. It fixes this by down-weighting easy examples with a focal loss. That experiment was run on CIFAR and SVHN, where the unlabeled pool is artificially balanced.
Our hypothesis is that under real clinical skew, the decisive failure is co-adaptation. The network's features and its pseudo-labels drift together, so the network's own confidence cannot detect its errors. Reweighting the loss does not address that. An independent view does, and the fixed-feature heart sound literature already supplies one. Heart Sound Classification Using Deep Structured Features builds a tree of fixed Haar wavelet filters, a scattering transform. It adds heart-cycle state statistics and power spectra, compresses them with PCA, and classifies with an SVM. It reports that this generalizes better than learned networks on small cohorts. Because its feature map never changes, its errors are structurally decorrelated from those of a network trained on raw audio.
The new element is the use of that frozen view as the arbiter of pseudo-labels, combined with prior-matched quotas for the minority class. Neither idea appears in the file in combination with self-training on medical audio.
Research Program
The learned view is the 1D convolutional network of Classification of Heart Sound Recordings Using Convolution Neural Network. That pipeline:
- low-pass filters the signal at 400 Hz;
- resamples to 2 kHz;
- cuts recordings into 2.5-second windows;
- averages window decisions into a recording-level decision.
The frozen view is the scattering, state-statistics and spectrum pipeline of Tschannen et al., which relies on a hidden semi-Markov segmentation of each beat into S1, systole, S2 and diastole. Its SVM uses the asymmetric misclassification cost of Heart Sound Classification Based on Temporal Alignment Techniques, which penalizes missed abnormal recordings more than false alarms. Each round, the frozen view's SVM is refit on gold labels plus currently accepted pseudo-labels. This is cheap, and its capacity is limited, so it can shift its boundary but cannot reshape its features to match the network.
Each round runs four steps:
- Score. Both views score every unlabeled recording. Scores are calibrated on the labeled set.
- Estimate the abnormal rate. We estimate the fraction of abnormal recordings in the unlabeled pool, $\hat\pi$, by re-estimating the class prior from the frozen view's calibrated scores. The frozen view is used for this because it is the less drift-prone of the two.
- Admit pseudo-labels. At round $t$, among recordings both views call abnormal, we admit the top-ranked $\lceil \rho_t \hat\pi N \rceil$ as abnormal. Here $N$ is the unlabeled pool size and $\rho_t \in (0,1]$ grows across rounds, so easy cases enter first. Normals are admitted at the matching quota $\lceil \rho_t (1-\hat\pi) N \rceil$. Any recording where the views disagree, or where beat segmentation fails, stays unlabeled.
- Update the network. Accepted labels enter a slimIPL-style cache. The network is fine-tuned continuously rather than retrained from scratch, using slimIPL's decaying dropout schedule.
The quota is what keeps the abnormal class from shrinking. The agreement rule is what keeps the quota from being filled with confident mistakes.
The main analysis is per-round pseudo-label precision and recall for each class. These can be measured because our unlabeled pool consists of recordings whose true labels we hide. The prediction is that precision on abnormal pseudo-labels falls round over round under single-model self-training and under co-training of two networks. Under frozen-view co-training it should stay flat.
A secondary arm repeats the design on stroke speech from the SONIVA cohort used by Physiologically-Informed Feature Analysis of Acquired Speech Disorders for Stroke Assessment. There the frozen view is the eGeMAPS acoustic set plus glottal-flow parameters. SONIVA is balanced, 125 patients against 125 controls, so we induce imbalance by subsampling patients by speaker. We read this arm as a test of whether the mechanism transfers, not as a clinical result.
Baselines
Naive reference. The CNN trained only on the labeled subset with plain cross-entropy and window majority voting, as in Ryu et al. Ryu et al. report a hidden-test challenge score of 79.5%, with 70.8% sensitivity and 88.2% specificity. That gap is exactly the failure we target.
Strongest supervised competitor. The cost-sensitive SVM on fixed features. We run two variants:
- the Tschannen pipeline, which reports a hidden-test mean accuracy of 0.812;
- the Gonzalez Ortiz features (frequency-envelope features plus beat-alignment distances), which report a hidden-test score of 82.4%.
These are the strongest competitors because they are exactly what our frozen view is. If self-training merely lets the network catch up to them, the loop adds nothing. Both reported figures come from the PhysioNet/CinC 2016 hidden test set. Our experiments use the public training set, so those figures orient us but are not targets to compare against. We re-run every pipeline under our splits.
Strongest semi-supervised competitor. FocalMatch-style self-training: the same CNN, the same cache and the same rounds, but with confidence-thresholded pseudo-labels and focal loss on the unlabeled term. We found no reported result on Lacuna for FocalMatch on heart sounds. Also included:
- plain slimIPL-style self-training with neither quota nor second view;
- single-shot pseudo-labeling with one teacher, one fixed threshold and one student.
Ablations isolating the key ingredient. The central ablation replaces the frozen view with a second CNN, with a different seed and kernel width, trained on the same raw audio. It keeps everything else identical, including the quota. A second ablation keeps the frozen view but drops the quota in favor of a fixed confidence threshold.
Fairness rules. All methods share the same splits, the same labeled subsets, the same number of rounds and network updates, and the same hyperparameter search budget. Tuning uses only the labeled sites' validation fold. Beating a competitor means a gain in mean of sensitivity and specificity on the held-out site that exceeds the seed-to-seed standard deviation over five seeds. That gain must come with sensitivity at least as high as the competitor's. A gain bought with specificity alone does not count.
Why not just…?
Why not just fix the loss, as FocalMatch does? Focal loss changes how much each pseudo-labeled example counts. It does not change which examples receive a wrong label. FocalMatch was only shown on unlabeled pools that are balanced by construction. Our setting has a skewed pool and a site shift, and a confidently mislabeled murmur keeps its wrong label however it is weighted. This is a real competitor, and the experiment settles it directly. If focal-loss self-training matches our method on abnormal pseudo-label precision and held-out sensitivity, the frozen view is unnecessary and we will say so.
Why not just use the fixed-feature SVM and skip deep learning? Tschannen et al. and Gonzalez Ortiz et al. suggest this is a strong position on small cohorts. Both depend on beat segmentation, which the file notes breaks under arrhythmia or weak signals. Our design must therefore show gains over the frozen view alone, not just over the naive CNN. We also report the frozen view self-trained by itself. If the network never beats the frozen view, the honest conclusion is that unlabeled data should go into the fixed-feature model, and that is a useful negative result for Gap 5 in the file.
Why not just co-train two neural networks? Agreement between two networks trained on the same audio may be enough on its own. If so, the frozen-feature argument is wrong. The two-network ablation tests exactly this. Our hypothesis predicts that the networks' errors grow correlated over rounds, and we measure the correlation of their errors on hidden-label recordings each round.
Why not just pretrain a self-supervised audio model and fine-tune? slimIPL reports that wav2vec 2.0 pretraining costs roughly 300 TPU-days, against 75–83 for slimIPL. A heart sound pool is also far too small to justify that pretraining. More importantly, fine-tuning on skewed labels still leaves the decision boundary trained on scarce abnormal examples. We include a small contrastive pretraining of the same CNN on the unlabeled pool, at matched compute, as a baseline, and it can also be combined with our loop.
Proposed Experiments
The primary benchmark is the public PhysioNet/CinC 2016 training set, split by recording site (populations a–f). In leave-one-population-out fashion, the held-out site supplies both the unlabeled pool, with labels hidden, and a disjoint test portion. The remaining sites supply gold labels at 5%, 10% and 25% of recordings, sampled with natural class proportions. This mirrors deployment: a new hospital, new hardware, no labels, unknown abnormal rate.
Metrics. All reported per held-out site and averaged:
- the challenge's mean of sensitivity and specificity;
- sensitivity at the specificity of the supervised SVM;
- area under the precision-recall curve for the abnormal class;
- per-round pseudo-label precision and recall per class;
- the error in the estimated abnormal rate $\hat\pi$ against the true rate.
Decisive checks. There are three.
- Our method must beat the frozen-view SVM, FocalMatch-style self-training and the two-CNN ablation on held-out sensitivity without losing specificity.
- Abnormal pseudo-label precision must stay roughly flat over rounds for our method while declining for the ablations. This is the direct test of the co-adaptation hypothesis.
- A stress test deliberately misestimates the abnormal rate by ±50% to show how much the quota mechanism depends on that estimate.
If the first check fails while the second succeeds, the frozen view is a good filter but a poor teacher. We would then shift to using it only as a filter on network-generated pseudo-labels. The SONIVA arm, with imbalance induced at several ratios, tests whether the glottal and eGeMAPS frozen view produces the same flat-precision pattern in speech.
References
- Iterative Pseudo-Labeling for Speech Recognition
- slimIPL: Language-Model-Free Iterative Pseudo-Labeling
- FocalMatch: Mitigating Class Imbalance of Pseudo Labels in Semi-Supervised Learning
- Heart Sound Classification Using Deep Structured Features
- Classification of Heart Sound Recordings Using Convolution Neural Network
- Heart Sound Classification Based on Temporal Alignment Techniques
- Physiologically-Informed Feature Analysis of Acquired Speech Disorders for Stroke Assessment