Adapting Ultrasound Lesion Detectors to a New Hospital Using Only Its Normal Scans

Ultrasound lesion detectors trained at one hospital tend to fail at the next. Much of that failure has a specific shape. The new site's scanners and sonographers produce harmless structures, such as rib shadows, gland tissue, and muscle boundaries, that the detector has never seen and flags as lesions. Fixing this usually means annotating lesion boxes at the new site, which is slow and needs experts. It also runs into a basic problem: any single hospital sees few lesions.

The proposal rests on an asymmetry. Every hospital has a large supply of scans from patients known to have no lesion, and labeling such a scan takes one bit of clinical information rather than a hand-drawn box. Any detection on such a scan is a false alarm. We propose to adapt a two-stage detector to a new site using a few such lesion-free sweeps. A two-stage detector first proposes candidate regions and then judges each candidate. We leave the first stage untouched so it keeps finding lesions. We change only how the second stage judges candidates, by comparing each one against a small memory of features taken from the site's own known-normal tissue.

Core Claim

Our claim is that, under a change of hospital, a large share of the extra false positives produced by an ultrasound lesion detector are repeated look-alikes. They come from the target site's particular scanners and particular scanning habits. A non-parametric memory of second-stage region features, taken from a handful of target-site lesion-free sweeps, should suppress them at fixed lesion recall. It should do better than three alternatives: recalibrating a global score threshold, fine-tuning the detector on the same negatives, and pseudo-label domain adaptation on unlabeled target video.

The two-stage split matters for a concrete reason. Because candidates are generated before they are judged, we can intervene where the error happens, in the judging, without risking the recall the proposer provides. A second claim, which we treat as an open empirical question rather than an assumption, is that these look-alikes depend partly on the individual operator and not only on the scanner. If so, a memory built from one sonographer's sweeps should transfer less well to a colleague at the same hospital than to that same sonographer's later scans.

How This Relates to Prior Work

Three groups of prior work bear on the proposal.

The ultrasound video detectors supply the setting and the closest mechanism. UltraDet suppresses false positives by attending to "negative temporal contexts," meaning earlier frames of the same video in which a candidate reveals itself as non-lesion tissue. It roughly halves the false-positive rate on the breast ultrasound video dataset CVA-BUS. CVA-Net introduced that dataset and a transformer video detector. Both are evaluated only on internal splits of one cohort. Our idea extends UltraDet's insight from negatives within one video to negatives across a new site's population of normal scans, and we evaluate across sites, which neither paper does.

Two papers from natural-image detection supply method and constraints. DeFRCN shows that in few-shot two-stage detection, gradients from the proposal network and the classification head conflict in the shared backbone. It stops the proposal network's gradients and adds a prototype-based calibration of the classifier. We reuse both ideas: a frozen proposal stage, and similarity-based rescoring with no gradients. DeFRCN, however, calibrates with positive class prototypes on natural photos, whereas we calibrate with site-specific negatives on ultrasound. D-adapt decouples cross-domain adaptation into separate category and box adaptors trained on pseudo-labels from unlabeled target images. This is the strongest prior answer to domain shift in detection. It relies on large unlabeled target sets, and its pseudo-labels can lock in mistakes when target data is scarce. Our negatives are known to be correct, so that failure mode does not arise.

The ultrasound representation and operator literature motivates the study. Anatomy-aware Representation Learning pretrains on 5.2 million images from 12 institutions, which raises the possibility that broad pretraining alone removes site shift. We test this directly by swapping in such a backbone. Its downstream evaluations use curated frames rather than the live search phase in which operators vary most. Work on operator dependence explains why our negatives should be specific to the site and the operator:

  • Pan Gao et al. note that their results come from a single expert sonographer.
  • Studies of standard-plane navigation (Dapueto et al., Guo et al.) locate the novice–expert gap in where the probe goes.
  • AutoDVT operated across three handheld scanners but was validated on only 51 patients at one center.

Research Program

We build the detector from a two-stage, real-time video detector of the UltraDet type, trained on a source site. At a target site we collect $K$ lesion-free sweeps, with $K$ ranging from 1 to 20. Some are full sweeps from lesion-free patients. Others are segments of patient videos that a clinician marks as containing no lesion. We pass these through the frozen detector and store the second-stage features of every proposal it generates, in a memory $M$. Every stored feature is a confirmed false alarm.

At test time, each candidate region $r$, with original score $s(r)$ and region feature $f(r)$, is rescored as

$$s'(r) = s(r) - \beta \, \max\left(0,\; \max_{m \in M} \cos(f(r), m) - \tau\right),$$

where $\cos$ is cosine similarity, $\tau$ is a similarity margin below which no suppression happens, and $\beta$ sets the strength of suppression.

The margin $\tau$ is chosen to limit harm to lesions. On source-site lesion boxes, it is set so that at most 1% of true-lesion features exceed $\tau$ against the target memory. Nothing in this design is trained on target data, and stage 1 is never modified. Recall can therefore drop only when a lesion closely resembles a stored normal structure, and we measure that drop directly.

The analysis has three parts. First, we decompose the cross-site increase in false positives into contributions from proposal (stage 1) and judging (stage 2). We do this by checking whether target-site lesions are still proposed, and whether false positives cluster tightly in feature space. Clustering is the condition under which a memory should work. Second, we vary $K$ to obtain a curve of adaptation cost against benefit. Third, we stratify target negatives by operator to test the operator-specificity hypothesis.

Baselines

The first baseline is doing nothing. The source-trained detector runs unchanged at the target site, with its operating threshold set on source validation data. This measures the raw cost of moving between sites.

The second is the strongest existing method on this data, UltraDet itself, retrained on our source split. Its in-distribution figures on CVA-BUS are FP90 of 5.7 sequences per minute and Pr90 of 90.8%, as reported in Mining Negative Temporal Contexts. FP90 is the lesion-level false-positive rate at 0.90 recall, and Pr90 is precision at that recall. For context, CVA-Net reports AP 36.1 and AP50 65.1 on its own split. We found no reported cross-site result on Lacuna for either detector, so the cross-site numbers for both will be ours. Our method is added on top of UltraDet. Beating UltraDet means reducing target-site FP90 by at least 25% relative while keeping recall within 1 point, across at least two held-out sites.

The third is global threshold recalibration. The detector's score threshold is reset per site using the same $K$ negative sweeps, so that its false-alarm rate on them matches the source rate.

The fourth is negative fine-tuning. The second-stage head is fine-tuned with the same negatives used as hard negatives, with gradients from the proposal network stopped, following DeFRCN's gradient decoupling.

The fifth is D-adapt. It is run with its cascaded pseudo-labeling on all unlabeled target video, which gives it strictly more target data than our method.

Every method uses the same source checkpoint, the same $K$ sweeps where applicable, and the same tuning budget: 20 configurations each, selected on a held-out target validation fold that is never used for testing.

Two ablations isolate the key ingredient. In the first, the memory is filled with an equal number of source-site negatives instead of target-site negatives. This tests whether specificity to the site is what matters. In the second, the memory is filled with randomly sampled target frames that include lesions, which tests whether the confirmed-negative label matters.

Why not just…?

Why not just recalibrate the threshold on the new site's normal scans? This is the most serious threat, because it uses the same cheap data and has one parameter. A global threshold trades recall for precision across all candidates at once. A memory removes only candidates that resemble stored normals. If the look-alikes form tight clusters, which the first analysis checks, the memory should dominate on the curve of false positives against recall. If threshold recalibration matches our method on that curve, the core claim is false, and we would report that the problem is score calibration rather than structured look-alikes.

Why not just fine-tune the second stage on those negatives? With five to twenty sweeps, fine-tuning can overfit and shift the classifier away from lesions. DeFRCN's results show that even decoupled fine-tuning needs care in the few-shot regime. Fine-tuning remains a real competitor at larger $K$. The sweep over $K$ will show where, if anywhere, fine-tuning overtakes the memory, and we expect such a crossover to exist.

Why not just run unsupervised domain adaptation such as D-adapt? D-adapt needs no labels at all. Its pseudo-labels, however, come from the very detector that is making the false positives, and scarce target data is exactly where confirmation bias appears. The direct comparison with D-adapt answers this. If D-adapt wins, the one-bit "no lesion" label is not worth collecting.

Why not just use a backbone pretrained on many sites? Pretraining across many institutions may already remove site shift. We swap in an A-ViT-style backbone where weights are available and repeat everything. If the gap between sites closes without the memory, our method is unnecessary for well-pretrained models.

Proposed Experiments

The data come from two sources, used in increasing order of realism.

The first is CVA-BUS, which contains videos from two scanner types (LOGIQ-E9 and Philips L9-3). If per-video scanner metadata can be recovered, we split by scanner as a proxy for a change of site, training on one scanner and testing on the other. Negatives come from lesion-free segments of target-scanner videos that are held out from testing.

The decisive cohort is a new multi-site breast ultrasound video collection: three or more hospitals, including at least one using handheld devices, with each sweep tagged by operator and with lesion-free patient sweeps recorded. Test lesions are box-annotated only for evaluation.

We report FP90 and Pr90, following UltraDet, as the primary metrics, together with AP, AP50, lesion recall, and frames per second, since the memory lookup must stay real-time.

Three checks settle the claims. The first is whether the memory beats threshold recalibration on the curve of false positives against recall at matched $K$. If it does not, the structured look-alike hypothesis fails. The second is whether the target-memory versus source-memory ablation shows a gap. If it does not, any gain comes from adding negatives in general, not from specificity to the site. The third is whether, within a site, memories from the same operator suppress more false positives than memories from other operators, at matched sweep count. A clear gap would show that operator habits, not only hardware, define what a hospital's false alarms look like. That would argue for collecting negatives per operator rather than per site.