Bedside lung ultrasound is fast, cheap and radiation-free. That makes it one of the few imaging tests that can inform triage within minutes rather than hours, and AI assistance has already been shown to raise non-expert reading accuracy from 68.1% to 93.4% in a resource-limited intensive care unit (Clinical benefit of AI-assisted lung ultrasound in a resource-limited intensive care unit). Expert labels for these videos are scarce, though, so the field has turned to self-supervised pretraining: the network first learns from unlabeled clips by predicting hidden parts of them, then is fine-tuned on a few hundred labeled frames. The trouble is what these unlabeled clips mostly contain. The diagnostic signal lives in a thin bright line, the pleura, and in the acoustic echoes beneath it. Most of the image is skin, fat and muscle above it. One supervised study found that a model trained on whole frames learned to read the thickness of the patient's subcutaneous fat, a proxy for obesity and therefore for COVID-19 risk, instead of reading the lung (The Role of Pleura and Adipose in Lung Ultrasound AI). Current self-supervised ultrasound encoders pretrain on exactly these unmasked frames.
We propose to pretrain lung ultrasound encoders in a coordinate frame anchored to the pleura. We find the pleural line automatically, without any manual masks, and resample each frame so the pleura becomes a horizontal line at a fixed row. The network then hides and predicts only content at or below that line, and it predicts learned feature targets rather than raw speckle. In two sentences: the encoder never gets rewarded for modeling the chest wall, so it should spend its limited capacity on the lung and carry less information about body habitus. We test whether this yields better severity scoring at small label budgets, and embeddings that add lung-specific information to bedside risk models rather than duplicating information already present in body mass index.
Core Claim
Restricting where self-supervised prediction happens, using an automatically detected and straightened pleural frame, improves label efficiency on lung ultrasound severity scoring. It also measurably reduces how much body-habitus information leaks into the representation. Our specific bet is that the restriction must happen during pretraining, not only at fine-tuning time. With a few hundred labels, fine-tuning cannot fully undo an encoder that already spent its capacity on the chest wall.
This claim sits between three lines of work.
- The motivating evidence. The Role of Pleura and Adipose in Lung Ultrasound AI shows the habitus shortcut and fixes it with handcrafted pleural segmentation. Area under the ROC curve rose from 0.6553 unmasked to 0.7742 masked. That work trains supervised models from scratch on 93 patients and uses no pretraining.
- The reused mechanism. RGMIM supplies region-guided masking. It restricts masked autoencoding to lung fields in chest X-rays and reaches 0.771 accuracy with 1% of labels, against 0.754 for standard random masking. It needs radiologist lung masks for every pretraining image and reconstructs raw pixels. We remove both requirements.
- The constraint on prediction targets. URFM argues that pixel reconstruction in ultrasound memorizes speckle, and instead predicts frozen semantic embeddings for masked patches. Its masking is random and organ-agnostic. Ultrasound Lung Aeration Map via Physics-Aware Neural Operators pulls the other way: lung diagnosis depends on reverberation artifacts that high-level abstractions may discard. We therefore treat the prediction target as an experimental variable, not a settled choice.
- The strongest prior result. Efficient Lung Ultrasound Severity Scoring Using Dedicated Feature Extractor (MeDiVLAD) pretrains a small vision transformer with DINO (self-distillation, where a student network matches a slowly updated teacher on augmented views) on full, unmasked sectors.
Research Program
Automatic pleural frame. The first component locates the pleura without labels. We start from the handcrafted pipeline of Gare et al.: Gaussian blur, a vertical-gradient Sobel filter, and a polynomial fit of the brightest near-horizontal ridge. We add a temporal cue that requires no annotation. Lung sliding makes the pleura the shallowest depth at which frame-to-frame motion changes character, and A-line reverberations repeat at integer multiples of the pleural depth.
We keep only frames where these cues agree, using them as confident pseudo-labels. On those frames we train a lightweight segmenter, which then labels the hard frames where consolidation or shadowing fragments the line and the handcrafted fit fails. This directly targets the cascading-error weakness of two-stage pipelines. We report detector failure rates by severity grade so the weakness is measured, not assumed.
Each frame is then resampled along the fitted curve into a straightened image. In this image, row zero is the pleura, rows above are chest wall and rows below are the lung. The same transform applies to convex and linear probes, which matters because public curvilinear data (COVIDx-US) and the linear-probe study of Gare et al. differ in geometry.
Pretraining objective. We pretrain a ViT-S/8, the same backbone as MeDiVLAD, on unlabeled straightened clips. The chest-wall band is cropped to a fixed small margin above the pleura, so fat thickness cannot be read off the band's size. Masking is drawn only from the pleural band and the region below it. Masked patches are predicted in feature space, with targets from an exponential-moving-average teacher in the style of DINO.
We add one physics-motivated auxiliary. For masked patches below the pleura, the model also predicts a coarse depth-intensity profile. A-line periodicity and B-line vertical persistence are visible in that profile, while patch-level speckle averages out. The auxiliary is our response to the Luna concern that semantic targets discard artifact physics. If it does not help, we drop it.
Habitus-leakage probe. The pleural detector gives, for free, the depth of the pleura below the skin in the original unstraightened frame. This is a direct, label-free measurement of chest-wall thickness. We fit a linear probe from frozen embeddings to this depth, and its explained variance becomes a quantitative leakage score for any encoder. This turns the Grad-CAM anecdote of Gare et al. into a number we can compare across methods. It also lets us stratify every downstream metric by chest-wall thickness tertile.
Clinical prediction arm. This arm is secondary and its feasibility is uncertain. The file contains no public dataset that links lung ultrasound video to hard outcomes, so we found no reported result on Lacuna for ultrasound-video-based outcome prediction.
Where a partner site can provide admission clips with early vitals and outcomes, we feed pooled frozen embeddings into a gradient-boosted tree model alongside vitals and body mass index. Gradient-boosted trees are the model family that dominates tabular bedside prediction (Machine learning model to predict mortality in patients with skin and soft tissue infection in emergency department; Machine learning score to predict in-hospital outcomes in patients hospitalized in cardiac intensive care unit). The question is incremental value. Does adding the image embedding improve discrimination after body mass index is already in the model? A habitus-heavy embedding should add little once body mass index is present. A lung-specific one should add more.
This replaces the hand-measured, often-missing echo parameters of the ADDICT-ICCU study, where E/e' was missing in up to 25% of cases, with something computed automatically from the raw clip. It is also far shorter-horizon than the 48-hour note-plus-EHR models such as A Multimodal Transformer.
Baselines
Naive reference. A ResNet or ViT trained from scratch on full frames with the same label budgets. We pair it with the masked supervised model of Gare et al. trained from scratch, which is the strongest label-hungry method on linear-probe data. Its reported AUC on the curated linear-probe dataset is 0.7742 with Pleural + Merlin's-space masking, and 0.7642 with straightening at an accuracy of 0.5348. We will rerun it on our splits rather than rely on these figures, since that dataset may not be shareable. Beating it means a higher AUC at every label fraction, and parity when using fewer labels.
Strongest existing method. MeDiVLAD: DINO-pretrained ViT-S/8 on unmasked frames with dual-level VLAD video pooling. It reports a video-level ROC-AUC of 0.936 and accuracy of 82.60% on the combined COVIDx-US plus CoCross cohort. At frame level it reports 82.47% accuracy after fine-tuning on 300 annotated frames, all from Efficient Lung Ultrasound Severity Scoring Using Dedicated Feature Extractor. CoCross is private, so we will reimplement MeDiVLAD and report it on the public COVIDx-US subset under our splits, alongside the published number for context.
To keep the comparison fair, our encoder and the reimplemented baseline share the same unlabeled pool, backbone, pretraining epochs, VLAD video head and hyperparameter search budget. Our model must beat it on COVIDx-US video AUC at the 300-frame label budget, with patient-level splits and confidence intervals that exclude zero difference.
URFM. We also include URFM's encoder, fine-tuned identically. It is the largest ultrasound foundation model in the file and claims 20%-label performance that beats competitors at 100%. Its weights may not be available; if they are not, we substitute BiomedCLIP-target masked modeling with random masking, which is the URFM recipe on our pool.
Key ablations. Four variants isolate the ingredients: our full method; the same objective with random masking over the unstraightened full frame; straightening without restricting where masking happens; and the full method without the depth-profile auxiliary. The comparison between full and random masking isolates the pleural anchor itself.
Why not just…?
Why not just mask at fine-tuning time on top of an existing encoder? Feeding pleura-masked inputs to a MeDiVLAD-style encoder is the real threat here. So is mask-weighted pooling of its features, as in CT triage (Computed tomography–based COVID–19 triage through a deep neural network using mask–weighted global average pooling) or breast ultrasound lesion embeddings (Lesion-Centric Latent Phenotypes from Segmentation Encoders for Breast Ultrasound Interpretability). Either may recover most of the gain. We run both as baselines at matched labels.
If they match our method on severity scoring and on the leakage probe, the claim about pretraining-time restriction is false. The contribution would then shrink to the label-free pleural detector and straightened frame, which we would report plainly. The leakage probe is what can separate them. Pooling-level masking hides chest-wall pixels, but the patch features themselves may still encode habitus through attention to the whole frame.
Why not just use URFM or another large ultrasound foundation model? URFM pretrained on about a million multi-organ images and reports strong 20%-label results. However, its masking is random over full sectors and its targets come from a vision-language encoder not trained on lung reverberation artifacts. Pleural "local bumps" are the early-disease signal Gare et al. emphasize. A model with fifty times more pretraining data could still win on raw scale. Running it at matched label budgets, and scoring it on the leakage probe, settles whether anatomical anchoring beats scale on this task.
Why not just add body mass index as a covariate, or adversarially remove fat thickness? Adding body mass index fixes the clinical-prediction confound but does nothing for severity scoring, where habitus is a pure shortcut. An adversarial head that penalizes predicting chest-wall depth is a cheap competitor to restricting pretraining. We include it as an extra baseline on top of DINO. If it lowers leakage as much and scores as well, it is the simpler answer and we should say so.
Why not just use clinical scores or a tabular model without imaging? ADDICT-ICCU already shows gradient-boosted tabular models far exceed bedside heuristics, at AUROC 0.88–0.90 versus 0.51–0.65. In early triage, however, the labs and echo measurements they rely on are often missing. The tabular-only model with body mass index is therefore the reference that the imaging arm must improve on, not something to argue away.
Proposed Experiments
Pretraining uses unlabeled clips from COVIDx-US plus any additional unlabeled lung ultrasound a partner can provide. Severity scoring is evaluated on COVIDx-US using its lung ultrasound score labels, with patient-level splits. If accessible, it is also evaluated on the linear-probe cohort of Gare et al. to test cross-probe transfer.
Label budgets are 1%, 5%, 20% and 100% of annotated frames, plus the 300-frame setting from MeDiVLAD. Metrics are frame and video ROC-AUC, accuracy and macro-F1, each with patient-bootstrap confidence intervals.
First decisive check: leakage. The pleural-depth probe should show clearly lower explained variance for our encoder than for DINO on full frames. Performance should also be flatter across chest-wall thickness tertiles. If leakage is unchanged, the mechanism story fails even if accuracy rises.
Second decisive check: label-efficiency curve. Our method should beat full-frame DINO and fine-time masking most at the smallest budgets, with the gap narrowing as labels grow. That shape is the signature of capacity allocation mattering only when labels are scarce.
Third check: detector robustness. We report pleural detection failure by severity grade, and rerun severity scoring with detector-failure frames excluded versus kept. This bounds how much the anchor hurts precisely in the sickest patients.
Clinical arm. Where linked outcome data exist, the primary metric is the change in AUROC and AUC-PR from adding image embeddings to a tabular model that already contains early vitals and body mass index. Each encoder gets its own comparison. A lung-specific representation should show a positive increment that survives adjustment for body mass index. A habitus-heavy encoder should show an increment that largely vanishes once body mass index is included.
References
- Clinical benefit of AI-assisted lung ultrasound in a resource-limited intensive care unit
- The Role of Pleura and Adipose in Lung Ultrasound AI
- RGMIM: Region-Guided Masked Image Modeling for Learning Meaningful Representations from X-Ray Images
- URFM: A general Ultrasound Representation Foundation Model for advancing ultrasound image diagnosis
- Ultrasound Lung Aeration Map via Physics-Aware Neural Operators
- Efficient Lung Ultrasound Severity Scoring Using Dedicated Feature Extractor
- COVIDx-US: An Open-Access Benchmark Dataset of Ultrasound Imaging Data for AI-Driven COVID-19 Analytics
- Machine learning model to predict mortality in patients with skin and soft tissue infection in emergency department
- Machine learning score to predict in-hospital outcomes in patients hospitalized in cardiac intensive care unit
- A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction
- Computed tomography–based COVID–19 triage through a deep neural network using mask–weighted global average pooling
- Lesion-Centric Latent Phenotypes from Segmentation Encoders for Breast Ultrasound Interpretability