Finding Crackles Without Timestamps: Recording-Level Labels on Respiratory Foundation Models

Respiratory sound models are now very good at one narrow job: sorting a short clip of a single breath into "normal," "crackle," "wheeze," or "both." The clips they sort were cut by hand. On the standard lung-sound benchmark, a clinician marked the start and end of every one of 6,898 breathing cycles before any model ran. Hospitals do not produce that kind of annotation. What they do produce is a label for the whole recording ("abnormal sounds present," "COPD") attached to tens of seconds of unsegmented stethoscope audio. This proposal asks whether a respiratory-specific pretrained encoder can turn those cheap, recording-level labels into accurate timestamps of where the abnormal sounds occur. It also asks the practical question: how many expert-minutes of dense event labelling does that replace?

The idea is to treat each recording as a bag of short, overlapping time windows. An encoder pretrained on hundreds of hours of unlabeled respiratory audio scores every window. The recording-level label is explained by the few windows that score highest. Domain pretraining is the core mechanism, and the expert annotation bottleneck defines the setting and the yardstick. The contribution is not "use multiple-instance learning (MIL) on audio." It is a controlled test of whether in-domain pretraining is what makes weak labels sufficient for localization. That test is paired with a cost-matched comparison against simply paying experts to label a few events.

Core Claim

Recording-level labels on a respiratory-pretrained encoder, pooled with a rule that matches clinical logic, should localize crackles and wheezes in continuous auscultation. The clinical logic is that one abnormal event makes the whole recording abnormal. The localization should be close to a detector trained on dense event timestamps. It should clearly beat the same pipeline built on general-audio or speech encoders. And for a fixed budget of expert time, it should beat a detector trained on a small densely labelled subset.

Two claims are new, and both could fail. The first is that the benefit of in-domain pretraining is larger for localization under weak labels than for classification of pre-cut cycles. The reason is that weak supervision gives the model no help in separating a 20 ms crackle from friction noise, so the representation has to do that work alone. The second is that stethoscope-device shortcuts, which Stethoscope-guided Supervised Contrastive Learning for Cross-domain Adaptation on Respiratory Sound Classification showed dominate representations even under full supervision, get worse when labels are coarse. Removing them should matter more here than it did in that paper.

Where This Sits

Pretrained encoders. The method reuses the open encoders of Towards Open Respiratory Acoustic Foundation Models: Pretraining and Benchmarking. OPERA pretrained contrastive and masked-reconstruction models on about 400 hours of breathing, cough and auscultation audio. It beat general audio models on 16 of 19 tasks. However, every auscultation evaluation used pre-cut segments and supervised probes. Our work adds no new pretraining corpus. It changes what the encoder is asked to do: find events, not only classify cuts.

Evidence that the gap exists. Two papers found evidence for the gap without closing it. The BioCAS 2023 winner, A Deep Learning Architecture with Spatio-Temporal Focusing for Detecting Respiratory Anomalies, trained separate models for event-level and recording-level tasks on the pediatric dataset SPRSound. It never used recording labels to find events. Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment (REACH) aligned OPERA with clinical text to avoid task labels. It reports failure on fine-grained acoustic distinctions and still assumes pre-cut inputs. That failure suggests coarse text or patient signals alone will not surface short events. We therefore use the weak label as a bag-level constraint rather than a text target.

Constraints on the design. Two papers limit which designs make sense. Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification argues that mixing a normal breath with a crackle does not give a "half-abnormal" sound, because any crackle makes the cycle abnormal. We take this as the justification for max-like pooling rather than averaging. The SG-SCL paper reports that adversarial device removal underfits small CNNs. So device removal here is applied only to transformer encoders, and it is ablated.

Methods from imaging. The MIL formulation is borrowed from weakly supervised pathology imaging, for example HistoSegNet: Semantic Segmentation of Histological Tissue Type in Whole Slide Images. There, patch tags yield fine tissue maps. The analogy to audio is direct: a long recording is a slide, and a transient crackle is a rare positive patch. The same literature warns about confirmation bias from pseudo-labels and about in-bag class imbalance. Both are addressed below.

Research Program

Each recording is sliced into windows of about 250 ms with 50% overlap. This is short enough that a crackle dominates its window, and long enough for a wheeze to show its pitch contour. Each window $x_t$ is encoded by a frozen or lightly fine-tuned respiratory encoder $f$. A small head $g$ outputs a per-class probability $p_{t,c}$ that window $t$ contains event class $c$.

The recording-level probability for class $c$ combines window scores with a smoothed "any event" rule. One option is noisy-OR:

$$P_c = 1 - \prod_{t=1}^{T} (1 - p_{t,c}),$$

where $T$ is the number of windows. Noisy-OR is the probability that at least one window is positive if windows were independent. We also compare it with a top-$k$ mean, because noisy-OR saturates on long recordings. Training minimizes binary cross-entropy between $P_c$ and the recording label. At test time, windows with $p_{t,c}$ above a threshold tuned on validation recordings are merged into events with onsets and offsets.

Three additions address the known failure modes of weak supervision.

The first is a device-adversarial branch. A classifier predicts stethoscope identity from window embeddings, and its gradient is reversed into the encoder, following SG-SCL. It is applied only at the recording level, with no cycle labels.

The second is a guard against confirmation bias. A second round of training uses high-confidence windows as instance pseudo-labels. It does so only if round-one localization on a small validation set with event timestamps improves, and otherwise the round is dropped.

The third is a temporal-resolution check. If OPERA's native input length blurs sub-50 ms crackles, we continue masked-reconstruction pretraining on the same unlabeled corpus with shorter patches. We then report whether this changes localization while leaving cycle classification unchanged.

Those are the components to build. The analysis produces a cost curve. Expert annotation time is estimated as a per-recording labelling time for weak labels and a per-event timestamping time for dense labels. Both are measured in a small timing study with one clinician, or reported as a sensitivity range if that is not possible. The weak-label system is then placed on the curve of a dense-label detector trained on increasing fractions of events.

Baselines

Reference points. The do-nothing reference assigns every window of a recording the recording's own label. This is the localization implied by a recording classifier alone, and it sets the floor for event-level precision. The "recording-level classification only" reference is the BioCAS winner's recording model. It reports a SPRSound Task 2-1 score of 0.744 in A Deep Learning Architecture with Spatio-Temporal Focusing for Detecting Respiratory Anomalies. Our recording-level output should at least match it, so that localization is not bought at the cost of diagnosis.

Strongest competitor: full event supervision. The strongest existing method is a detector trained on dense event labels. We use the same BioCAS architecture, which reports Task 1-1 = 0.810 and Task 1-2 = 0.667 on SPRSound events. It is the strongest competitor because it sees exactly the timestamps we withhold. Fairness is handled two ways. We compare against the paper's reported challenge-protocol numbers. We also rerun the architecture ourselves on the identical train/test split, with the same compute and hyperparameter search budget as our method. Beating it is not the goal. Recovering at least 90% of its event score with zero event labels counts as success, and matching it at equal expert-minutes counts as a clear win.

Pretraining ablation. This isolates the key ingredient. The identical MIL pipeline, windows and tuning budget are run on three encoders: an AudioSet-initialized Audio Spectrogram Transformer (a transformer over spectrogram patches, as in MixCL), a wav2vec 2.0 speech encoder, and OPERA-CT/GT. The claim requires the OPERA gap on event localization to exceed its gap on recording classification. Otherwise pretraining helps diagnosis but not the annotation bottleneck.

Cycle-classifier transfer. An automatic breath segmenter based on signal envelope is combined with a cycle classifier trained on ICBHI cycles, using MixCL's recipe, which reports a 62.37% 4-class ICBHI score in Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification.

Component ablations. We remove the device branch, swap noisy-OR for mean pooling, and drop the second pseudo-label round.

Why not just…?

Why not just have experts label a few hundred events and train a supervised detector? This is the real threat. If a dense detector trained on 5–10% of SPRSound events already reaches the full-supervision score, weak labels buy little. We do not argue around this; the cost curve settles it. If the dense-label curve crosses our method at fewer expert-minutes than labelling all recordings, we will report that plainly. We would then reframe the method as a way to choose which recordings get dense labels: windows it flags as uncertain are sent for timestamping first.

Why not just use a general audio foundation model with the same MIL head? OPERA's 16-of-19 win over general models and REACH's zero-shot advantage (61.3% mean AUC versus 51.4% for CLAP and 54.9% for Qwen2-Audio, REACH) concern classification only. Whether that advantage carries over to localization is exactly the pretraining ablation. If AudioSet-AST localizes as well as OPERA, the "domain pretraining is the mechanism" claim fails, and we will say so.

Why not just segment breaths automatically and reuse the best cycle classifier? This labels whole cycles, not events within cycles. It also inherits ICBHI's device mix, which SG-SCL showed the representation encodes. The cycle-classifier transfer baseline tests it directly on SPRSound, a different population and device. We expect it to lose mainly on onset/offset precision and on cross-dataset transfer.

Why not just "heal" abnormal spectrograms into normal ones and subtract, as PathoGAN does for MRI? Pathology Segmentation using Distributional Differences to Images of Healthy Origin reached about 72.9% Dice from binary labels, against 86.3% fully supervised. That is a 13-point gap and comes with known boundary errors. For transient, broadband crackles, a translation model is more likely to smear energy than to remove events cleanly. It is a heavier model with an adversarial training loop. We treat it as a secondary comparison only if MIL localization proves poor.

Proposed Experiments

Primary setting. The primary setting is SPRSound, because it uniquely provides both recording labels and event timestamps for the same 2,683 recordings. We train only on recording labels and score events with the BioCAS Task 1 metrics. We also report event-based F1 at onset tolerances of 50 ms and 200 ms.

Transfer setting. The second setting is ICBHI. Here recording labels are derived as "any cycle contains crackle/wheeze." Localization is evaluated against the cycle timestamps and labels, and the four stethoscopes allow a leave-one-device-out test of the device branch. ICBHI audio was in OPERA's unlabeled pretraining pool. We therefore also report SPRSound results, which OPERA did not pretrain on, as the clean transfer check.

Decisive checks. Four results decide the outcome. First, OPERA's localization margin over AudioSet-AST must exceed its classification margin. Second, the weak-label event score must reach at least 90% of the rerun dense-supervision score. Third, the leave-one-device-out gap must shrink with the device branch. Fourth, the dense-label cost curve must fall below our method's point only at a budget larger than weak labelling costs. Failing the first check reframes the paper as a negative result on domain pretraining. Failing the fourth reframes it as an annotation-triage method.