Re-speckling, Not Despeckling: Matching Scanner Speckle for Locked Ultrasound and OCT Models

Ultrasound and optical coherence tomography (OCT, an optical analogue of ultrasound used for retinal imaging) produce images covered in speckle. Speckle is a grainy interference pattern whose grain size and shape are set mostly by the scanner's point spread function, meaning how the device blurs a single point. A segmentation or detection model trained at one hospital therefore learns that hospital's speckle along with the anatomy. When the same model is deployed on another vendor's probe, the grain changes and accuracy can collapse. Certified clinical models usually cannot be retrained on site. The common remedy is to despeckle incoming images, but a locked model never saw clean images during training, so a despeckled image is also out of distribution for it. We propose the opposite operation. For each incoming image we estimate the tissue signal and the local tissue scattering statistics, strip out the target scanner's speckle correlation, and re-synthesize speckle with the source scanner's correlation and log-compression. The locked model then sees images that look statistically as if they came from the scanner it was trained on.

The method has no adversarial network and needs no target labels. At deployment it does not need source images either, only a few physical parameters describing the source scanner. Because re-synthesized speckle is random, we can run the frozen model on several speckle draws per image and average them, which also gives a per-pixel uncertainty estimate. The central question is empirical and falsifiable: does physically matched re-speckling beat despeckling, generic noise injection and unpaired image translation as an input-side fix for a model whose weights cannot change?

Core Claim

Our claim is that much of the cross-scanner failure of locked coherent-imaging models comes from a shift in the second-order statistics of speckle. Second-order statistics describe how neighbouring pixels co-vary, which sets the size and anisotropy of the speckle grain. The claim is that correcting this shift physically on the input side recovers more accuracy than removing speckle.

We model the displayed image at position $r$ as

$$y(r) = C_{\theta}\big(\,|(h * (\sqrt{x}\, w))(r)|\,\big),$$

where $x$ is the tissue reflectivity, $w$ is circular complex Gaussian scatterer noise, $h$ is the scanner's point spread function, $*$ is convolution, and $C_\theta$ is the log-compression and gain curve with parameters $\theta$. The scanner identity lives in $(h, \theta)$. The tissue lives in $x$ and in a local Nakagami shape map $m(r)$, which measures how far local scattering departs from fully developed Rayleigh speckle and carries real diagnostic information about scatterer density.

The adapter computes

$$\tilde y = C_{\theta_s}\big(|h_s * (\sqrt{\hat x}\, w_{\hat m})|\big).$$

Here $\hat x$ is the tissue reflectivity estimated from the target image, and $\hat m$ is the local shape map estimated from the target image. The draw $w_{\hat m}$ is a scatterer field whose local statistics follow $\hat m$. The pair $(h_s, \theta_s)$ holds the source scanner parameters, fitted once from the source training set. Scanner-dependent texture is therefore replaced, while the tissue's scattering character is kept.

Relation to Prior Work

Several papers motivate the gap by showing that the problem is real and is not yet solved on the input side.

AMOS benchmarks multi-organ segmentation across scanners and reports a drop of more than 20% in mean Dice on unseen MRI vendors, which shows how fragile models are across scanners outside CT.

Domain Adaptation via CycleGAN for Retina Segmentation in OCT translates between UK Biobank and noisy Topcon RETOUCH scans with a segmentation-guided CycleGAN. It reaches 0.937 Dice against 0.349 without adaptation and 0.899 with mathematical noise injection. That 0.899 figure matters for us. Generic noise injection recovers most of the gap, which suggests texture statistics are the main lever, while a remaining gap is left for something better matched. Their method differs from ours in three ways: it requires source and target images together during adversarial training, it has documented boundary misalignment failures, and it does not model speckle formation.

ToMo-UDA goes the other way from us. It ignores texture and aligns anatomical topology graphs across fetal ultrasound centers, retraining the detector to 71.33% mAP against 43.90% source-only. It acknowledges that its bounding boxes absorb speckle clutter. We fix texture and leave the detector untouched, so the two methods are complementary rather than competing.

INPUTADAPTER establishes the "adapt the data, not the model" regime for locked clinical predictors on intensive-care time series. It uses target labels and learned transformations. We bring the regime to coherent images and replace learned transformations with a physical parametric one that needs no target labels.

Other work supplies the method. Speckle2Self despeckles ultrasound self-supervised through multi-scale consistency, transfers zero-shot from Clarius to Siemens Juniper and Cephasonics scanners, and raises SAM2 carotid segmentation AUC from 0.945 to 0.985. We reuse it, trained on unlabeled target frames, as the estimator of $\hat x$. Speckle2Speckle uses Field II simulations with fixed geometry and independent scatterer draws, and we reuse that simulation protocol for controlled experiments. The whitening idea in deep PACO, which normalizes local speckle covariance before detection, is the closest conceptual ancestor of matching second-order statistics. Their goal is a stationary background for a detector trained on whitened data. Ours is to reproduce a specific scanner's covariance for a detector we cannot touch.

Two papers constrain the design. Is Speckle Noise More Challenging to Mitigate Than Additive Noise? argues that speckle is multiplicative and spatially correlated. That rules out the Gaussian-likelihood shortcut taken by MRI-derived normalizing flow priors, which also require slow per-patch optimization. DRGAN and HDcycleGAN produce clean OCT output within a single device. They show despeckling can look good, but they never test whether a downstream model trained on speckled data benefits, and that is the question we pose.

Research Program

The work has three parts.

First, we build a scanner parameter estimator. It fits $h$ from the speckle autocorrelation (axial and lateral full width at half maximum, plus center-frequency modulation) in regions it detects as homogeneous. It fits $\theta$ from the envelope histogram under a Rayleigh or Nakagami model. It also produces the per-pixel map $\hat m$. We validate the estimator on Field II phantoms with known point spread functions.

Second, we assemble the re-speckling adapter. It chains the target-trained Speckle2Self estimate $\hat x$, deconvolution of the residual correlation, and resynthesis with $(h_s, \theta_s)$. It runs K draws, with K between 1 and 8, and averages the frozen model's predictions over them. The adapter has no trainable parameters beyond those of the despeckler.

Third, we test when the idea fails. We measure how much of the shift is texture and how much is anatomy or protocol by adding ToMo-UDA-style anatomical alignment on top of the adapter where retraining is allowed. We also check whether keeping $\hat m$ matters for pathology, for example drusen in OCT and plaque in carotid scans.

An optional variant adds a small tuning of $(h_s, \theta_s)$ at test time to minimize the frozen model's prediction entropy. We report it separately, because it makes the adapter depend on the model.

Baselines

The naive reference point is the locked source model applied to raw target images. The file reports 0.349 Dice for this setting on UK Biobank with the RETOUCH-trained LF-UNet (Chen et al.) and 43.90% mAP for source-only transfer on FUSH² fetal heart (ToMo-UDA). Any method must clearly exceed these.

The first texture baseline is first-order intensity matching: histogram matching of target images to the source intensity distribution. It costs nothing and corrects gain and compression but not grain. We run it on the identical target pool.

The second baseline is generic noise injection toward source appearance, as in Chen et al., who report 0.899 Dice on the OCT task. We reimplement it in two forms: uncorrelated multiplicative noise, and speckle simulated with a default point spread function rather than the fitted one. The fitted-versus-default comparison isolates the value of estimating $h_s$.

The third baseline is despeckling alone: Speckle2Self applied to target images and fed directly to the frozen model. It shares our estimator of $\hat x$ exactly, so the only difference from our method is the resynthesis step. This is the ablation that isolates our key ingredient.

The strongest existing input-side method is the segmentation-guided CycleGAN of Chen et al., with 0.937 Dice and 0.978 pixel accuracy on UK Biobank. We retrain it on our splits with the same unlabeled target pool. It is allowed source images, which we are not allowed.

On FUSH², the strongest reported method is ToMo-UDA at 71.33% mAP, with SIGMA++ at 57.83%. Both retrain the detector, so they serve as reference ceilings for a setting where weights can change, not as like-for-like competitors.

Beyond the ToMo-UDA figures, we include test-time adaptation of batch-normalization statistics as a practical ceiling for methods that change the model, although that violates the locked setting. No reported result was found on Lacuna for batch-norm adaptation on these datasets, so we compute it ourselves.

Fairness rules apply to every method. All methods use the same frozen predictor, the same unlabeled target images, and the same small labeled target validation set, used only to select hyperparameters. Each method gets an equal hyperparameter search budget, at most 20 configurations. The OCT claim is beaten if re-speckling exceeds noise injection (0.899) by a margin larger than the variance across seeds and approaches or matches 0.937 without adversarial training and without source images at deployment. The ultrasound claim is beaten if re-speckling beats Speckle2Self despeckling on the frozen model's downstream metric.

Why not just…?

Why not just despeckle the target images? This is the most serious threat, because despeckling already helps SAM2 on carotid scans (Speckle2Self). But SAM2 was not trained on one scanner's speckle, whereas a locked task model was. Our prediction is that despeckling helps texture-agnostic models and hurts or under-helps models trained on speckled source images. The shared-estimator ablation tests exactly this. If despeckling matches re-speckling on the locked models, the resynthesis step is unnecessary and we will report that. The paper then becomes a negative result about the texture hypothesis.

Why not just histogram-match or inject noise? Both are real, cheap competitors, and noise injection already reaches 0.899 on OCT (Chen et al.). Neither corrects spatial correlation, which the speckle theory in Is Speckle Noise More Challenging… identifies as the defining difference from additive noise. The comparison between the fitted and default point spread function tests directly whether correlation matching adds anything beyond these two.

Why not just use the segmentation-guided CycleGAN? It works well at 0.937 Dice. However, it needs source data co-located with target data during training, which is often blocked by privacy rules. It learns an unconstrained mapping that can misplace boundaries, as its own error cases show. Its behaviour on each new scanner must also be validated again. If it clearly beats us, our remaining argument is deployability, and we will say so. We will also report a boundary-error analysis on pathological cases to check whether its failures concentrate there.

Why not just fine-tune or run test-time adaptation? On locked certified devices, changing weights triggers re-certification (INPUTADAPTER). We still run batch-norm adaptation to measure how much accuracy the locked constraint costs.

Proposed Experiments

The decisive controlled experiment uses Field II. We simulate matched anatomy (carotid and cardiac phantoms) under a grid of point spread functions and compression settings. We train a segmenter at one setting, freeze it, and test on the others. Because anatomy is identical across settings, any drop is purely scanner-induced. Re-speckling should recover close to the in-domain Dice. Despeckling should recover less. The fraction of the drop recovered should fall as the parameter estimates are deliberately perturbed. If recovery does not track estimation error, the mechanism is not what we claim.

On real OCT, the RETOUCH to UK Biobank task, we report Dice and pixel accuracy against 0.349, 0.899 and 0.937.

On real ultrasound, we use Clarius to Siemens Juniper and Cephasonics carotid data from the Speckle2Self setting, annotating a small evaluation set because no cross-scanner segmentation number is reported. We also use FUSH² Center 1 to Center 2 with a frozen detector, reporting mAP against 43.90%, and 57.83% and 71.33% as retrained references. We further test ToMo-UDA combined with re-speckling to check whether the gains add up.

Across all settings, we measure expected calibration error of the K-draw average against single-draw predictions and against temperature scaling. This tests whether variance across speckle draws is useful uncertainty, in the spirit of the augmentation-based calibration of Ouyang et al.. Their MRI figures (ECE falling from 14.48% to 12.78%) are not directly comparable to ours.

Finally, we run a pathology check. We compare lesion-level recall with and without the $\hat m$ map to test whether keeping tissue scattering statistics protects small diagnostic structures, the regime where AMOS and Speckle2Self both flag losses.