Hardware Shift as a Fitted Spatiotemporal Degradation

When a medical imaging model trained on a hospital's cart-based ultrasound machine is used on a cheap handheld probe, its accuracy collapses. On spine ultrasound, a detector trained on a cart system gains about 36 F1 points once the handheld images are corrected, which shows how much it loses without correction (Training-Free Image Style Alignment for Domain Shift on Handheld Ultrasound Devices). The usual fixes are heavy. One family learns a translator between devices. Another adapts millions of network weights. Both need a lot of target-device data, and hospitals rarely have it. We propose a lighter approach borrowed from satellite remote sensing. There, a coarse daily sensor such as MODIS is routinely treated as a blurred, rescaled, noisier version of a fine but infrequent sensor such as Landsat. Handheld devices also sit at a different point on the physical trade-off between spatial detail and frame rate: they fire fewer beam lines, use lower frame rates or frame averaging, and apply different intensity compression. Our bet is that most of the cart-to-handheld shift can be described by a small degradation operator with roughly a dozen physical parameters: a spatial blur kernel, a resampling factor, an intensity curve, a speckle-noise level, and temporal resampling plus persistence.

The proposal has three steps. First, fit those dozen parameters from a handful of unlabeled target-device videos (five to ten patients) by matching simple physical statistics: spatial power spectra, temporal autocorrelation and intensity histograms. Second, run labeled source videos through the fitted operator. Third, train or re-train the task model on the result. Estimating twelve numbers instead of millions of weights is what makes this workable with a tiny target sample. Treating space and time jointly is what makes it more than a blur augmentation. The resolution trade-off direction supplies the structure of the operator, cross-device generalization is the application, and small-sample overfitting is the binding constraint that motivates keeping the adapted part tiny.

Core Claim

A physically parameterized, jointly spatial-and-temporal degradation operator, fitted from very few unlabeled target-device subjects, closes most of the cross-device performance gap on dynamic imaging tasks. It should do better than three alternatives: randomized augmentation over the same operator family, generative style translation, and per-video restoration. We measure "most of the gap" as the fraction closed between a source-only model and a model trained on labeled target data.

The fitting objective is deliberately low-capacity:

$$ \hat\theta = \arg\min_{\theta} \sum_{s} w_s \,\big\| \phi_s\big(D_\theta(X_{\mathrm{src}})\big) - \phi_s(X_{\mathrm{tgt}}) \big\|^2 . $$

Here $D_\theta$ is the degradation operator with parameters $\theta$ (about 10–15 of them), and $X_{\mathrm{src}}$ and $X_{\mathrm{tgt}}$ are source and unlabeled target video clips. Each $\phi_s$ is a fixed, hand-defined statistic: the radially averaged spatial power spectrum, the frame-to-frame intensity autocorrelation, or the grey-level histogram. The weight $w_s$ normalizes each statistic's scale.

Because $\theta$ is so small, we can also approximate a posterior over it by simple rejection sampling. We then train on degradations drawn from that posterior rather than from a single point estimate. This gives a clean middle ground between "fitted" and "randomized" augmentation, and we test it explicitly.

Relation to Prior Work

The closest existing idea comes from remote sensing. A Sensor Bias Correction Method for Reducing the Uncertainty in the Spatiotemporal Fusion of Remote Sensing Images learns a MODIS-to-Landsat bias with gradient-boosted trees on neighbourhood statistics before running classical fusion. This motivates our proposal, since it shows that removing sensor bias before fusion helps: STARFM's correlation rises from 0.7990 to 0.8406 on Coleambally. It is also what we depart from. Their bias model is non-parametric and learned from abundant matched pixels, while ours is physical, low-dimensional and fitted from a few unmatched subjects.

Classical fusion models such as An enhanced unmixing model for spatiotemporal image fusion and A Rigorously-Weighted Spatiotemporal Fusion Model with Uncertainty Analysis supply the method. They treat coarse pixels as degraded mixtures of fine ones, and we reuse that modelling stance. They do not address label-scarce downstream tasks or medical devices.

On the medical side, two works attack device shift with learned machinery. TISA (Training-Free Image Style Alignment for Domain Shift on Handheld Ultrasound Devices) aligns handheld images to the cart style with a frozen conditional diffusion model and uncertainty filtering. The C3 model (Domain and Geometry Agnostic CNNs for Left Atrium Segmentation in 3D Ultrasound) uses an adversary that tries to predict the vendor, plus a shape prior. Both operate on static frames or volumes. TISA also needs a diffusion model trained on large source data and many sampling passes per test image. We ask whether an explicit physical model captures the same shift more cheaply. We also ask whether it captures the temporal part of the shift, which neither method touches. Optical Coherence Tomography Harmonization with Anatomy-Guided Latent Metric Schrödinger Bridges constrains our design. It shows that unconstrained translators hallucinate anatomy: the DDIB translator's retinal-boundary error of 0.89 pixels is more than twice their method's 0.43. A degradation operator made of blur, resampling and intensity maps cannot invent structures, and we argue this is a safety advantage for small-sample regimes.

Deep Ultrasound Prior (Blind Restoration of High-Resolution Ultrasound Video) works in the opposite direction: it restores low-resolution echo video per clip using test-time optimization. Its EchoNet-Dynamic result is our main quantitative reference. Finally, the small-sample literature supports the premise that adapting few parameters beats adapting many. FEDD finds that frozen features with shallow probes beat end-to-end fine-tuning at 5–10% labels, and Thrifty Neural Architecture Search finds that a 0.61M-parameter network beats UNet++. MR-JEPA shows that a spatiotemporal encoder trained on a single vendor's scanners gives strong frozen features. We use it as a probe backbone in the cardiac MRI extension, where its single-vendor pretraining is exactly the weakness our operator targets.

Research Program

The operator comes first. For ultrasound it acts in beam (polar) coordinates, which is where handheld hardware actually cuts corners. It contains an anisotropic Gaussian blur along the axial and lateral directions, a lateral line-density downsampling factor, a parametric log-compression and gain curve, a multiplicative Rayleigh-distributed speckle term, a frame-rate resampling ratio, and an exponential frame-persistence coefficient. Frame persistence averages successive frames, so it trades temporal sharpness for noise. For satellite data the same template reduces to the familiar point spread function plus per-band gain and offset.

Everything in the operator is differentiable. Every component preserves label geometry up to known resampling, so segmentation masks and keypoints are warped with the same resampling, and scalar labels such as ejection fraction are unchanged.

Fitting follows the objective above. We use a few hundred gradient steps, or rejection sampling for the posterior variant, using only unlabeled target clips from $k \in \{1, 3, 5, 10\}$ subjects plus the full target set as an upper bound. The task model is then trained on source data passed through the fitted operator. We run two variants: a small network trained from scratch, and a shallow probe on a frozen backbone.

The analysis component asks how much of the gap each operator component closes. We turn components on one at a time: spatial only, temporal only, intensity only, and jointly. This tests the central hypothesis that the spatial–temporal coupling matters. If the spatial-only operator does as well as the joint one, the connection to the resolution trade-off is decorative, and we will say so.

Baselines

The naive reference is the source-only model applied unchanged to target-device data. Next to it we report a model trained on labeled target data, which defines the gap. Every other number is expressed as the fraction of this gap closed.

The most dangerous competitor is randomized degradation augmentation. This is the same operator family with parameters drawn from broad hand-set ranges, trained for the same number of steps on the same source data. It is the obvious thing a practitioner would do. It needs no target data at all, so if it matches the fitted operator, the proposal's contribution vanishes. We give it the same tuning budget used for our method's hyperparameters, including the chance to widen or narrow the ranges using validation data from the source device only. Beating it means closing at least ten percentage points more of the gap on average across tasks, with confidence intervals from five seeds that do not overlap.

The strongest existing methods differ by benchmark. On the EchoNet-Dynamic low-resolution setting, the reference is Deep Ultrasound Prior, which reports an ejection-fraction mean absolute error of 7.23% against 10.26% for bicubic upsampling (Blind Restoration of High-Resolution Ultrasound Video). Following that paper's protocol, we train the EF predictor on high-resolution source video and evaluate on low-resolution target video. Beating it means an EF error below 7.23% at a small fraction of its per-video compute, which we measure in wall-clock time.

On the static cart-to-handheld ultrasound benchmarks, the reference is TISA. It reports 75.75% F1 for vertebral structure detection on the Clarius C3 spine data and 82.49% Dice for carotid intima-media segmentation on the Clarius L7 data (Training-Free Image Style Alignment for Domain Shift on Handheld Ultrasound Devices). Here we use only the spatial and intensity parts of the operator, fitted on at most ten target subjects. Matching TISA within two points while using no diffusion model counts as success, and exceeding it counts as a strong result.

For test-time adaptation, TISA reports beating TENT and CoTTA by over 14 F1 points on the spine data. We found no absolute TENT or CoTTA figure on Lacuna, so we rerun both with the batch sizes TISA describes.

On multi-vendor 3D echocardiography, the reference is the C3 adversarial and shape-prior model. It reports a mean surface distance of 2.01 mm, against 3.56 mm for a plain V-Net, when training on Philips EPIQ 7C and testing on GE Vivid E9 (Domain and Geometry Agnostic CNNs for Left Atrium Segmentation in 3D Ultrasound).

For remote sensing, the reference is gradient-boosted sensor bias correction followed by STARFM. It reports a correlation coefficient of 0.8406 and RMSE of 0.1416 on Coleambally, and 0.8316 and 0.1204 on Gwydir (A Sensor Bias Correction Method for Reducing the Uncertainty in the Spatiotemporal Fusion of Remote Sensing Images). We rerun it with the same number of matched date pairs our operator uses. That comparison, at $k = 1$ or 2 pairs, is the fair small-sample test.

The key-ingredient ablations are the spatial-only versus temporal-only versus joint operator, and the fitted point estimate versus posterior sampling versus the broad prior. The last comparison is the randomized baseline viewed as an ablation.

Why not just…?

Why not just use heavy random blur, noise and frame-dropping augmentation? This is the real threat, and the proposal is built to settle it. Our argument is that broad random ranges spend most of their training signal on degradations the target device never produces, and dilute the ones it does. The temporal persistence parameter matters most here, since frame averaging changes motion blur in a way that random frame dropping does not mimic. The fitted-versus-prior ablation at matched training budget decides the question. If the prior wins or ties, we will report the lesson that physics-structured augmentation, not fitting, is what matters, and that lesson would still be useful.

Why not just use generative style translation such as TISA, applied frame by frame? TISA works well on static images, but applying a diffusion model independently to each frame has no mechanism to keep speckle and boundaries consistent over time. The Schrödinger-bridge work shows that unconstrained translators shift anatomy (Optical Coherence Tomography Harmonization with Anatomy-Guided Latent Metric Schrödinger Bridges). On video, we will test frame-wise TISA-style alignment against our operator and measure frame-to-frame boundary jitter as well as task accuracy. If per-frame translation matches our accuracy and jitter, our advantage reduces to compute cost.

Why not just restore the target video with super-resolution before inference? Restoration attacks the same resolution gap from the target side. Deep Ultrasound Prior shows it works, reducing EF error from 10.26% to 7.23% (Blind Restoration of High-Resolution Ultrasound Video). However, it needs iterative optimization on every video, and it can only recover detail the model guesses. Our operator moves the source toward the target once, at training time, and costs nothing at inference. The EchoNet-Dynamic comparison reports both error and per-video compute, so a reader can judge the trade.

Why not just fine-tune on a few labeled target patients? Labels on new devices are exactly what is scarce, and the small-sample evidence in FEDD and Thrifty NAS indicates that updating many weights on tens of samples overfits. We nonetheless include full fine-tuning and probe-only fine-tuning with the same $k$ target subjects, now labeled, as an upper-cost comparison. Our method uses those subjects unlabeled. If a few labels beat our label-free fitting by a wide margin, that bounds how useful label-free fitting is in practice.

Proposed Experiments

The decisive experiment is the EchoNet-Dynamic resolution-shift setting. There we sweep the number of target subjects used for fitting over 1, 3, 5 and 10, and report EF mean absolute error for source-only, bicubic, randomized augmentation, Deep Ultrasound Prior, and our spatial-only, temporal-only and joint operators.

The prediction is specific. With five subjects, the joint operator should reach EF error below the 7.23% reported for Deep Ultrasound Prior. The temporal components should account for a measurable share of the improvement beyond spatial-only. The fitted operator should beat randomized augmentation by the margin stated above.

The static handheld benchmarks from TISA test whether a physical operator captures a true hardware shift rather than a synthetic downsampling. This matters because EchoNet-Dynamic's low-resolution videos may be closer to pure resampling than real device differences are. Those benchmarks are the Clarius C3 spine data and the Clarius L7 carotid data. If they are not obtainable, we fall back to the GE and Philips 3D echo comparison against C3.

The Coleambally and Gwydir experiments check that the fitting procedure transfers back to its original domain under a one- or two-pair budget. Gwydir's flood events deliberately stress the assumption that the shift is only a degradation.

Across all benchmarks we report two diagnostics. The first is the fraction of the gap closed. The second is the residual mismatch in the held-out statistics $\phi_s$ after fitting. A large residual, together with a large remaining gap, would be direct evidence that the dozen-parameter family is too small for that device. That would be a publishable negative result, and it would tell practitioners when they genuinely need a learned translator.