A common way to train tumor segmenters with scarce data is to paint synthetic tumors into healthy scans with a generative model and train on the result. This proposal starts from a suspicion about that recipe. Generative models leave faint high-frequency "fingerprints", meaning fine-grained texture statistics that real scanners never produce. When the synthetic content covers only the tumor region, those fingerprints sit exactly where the label says "tumor". A segmentation network can then learn a cheap rule, "generator texture means tumor", instead of learning what tumors look like. Such a model would miss real tumors. It would also flag healthy tissue on scanners whose native noise or sharp reconstruction filters happen to resemble the fingerprint.
We propose to test this shortcut directly and remove it at its source. The fix trains the generator's image decoder with a loss that compares images both pixel by pixel and in the frequency domain, and it adds a term that forces the local high-frequency spectrum inside each synthetic lesion to match the real tissue around it. The core claim fits in two sentences. In lesion-inpainting augmentation, generator fingerprints become a label-correlated cue rather than just a global domain gap. A training-time, spatially local frequency objective removes that cue more cleanly than post-hoc spectral cleanup or blunt blurring, which improves real-tumor segmentation and reduces scanner-dependent false positives.
What prior work established, and what it leaves open
Three findings motivate the project. Rethinking Bias in Generative Data Augmentation for Medical AI: a Frequency Recalibration Method (FreRec) shows that GAN and diffusion images carry high-frequency fingerprints that make synthetic augmentation fail. It repairs them after generation by replacing high-frequency content using retrieved real images, followed by a denoising autoencoder. It works only on whole synthetic images, it is evaluated only on classification, and the authors name in-training frequency constraints as open. Optimizing Synthetic Data for Enhanced Pancreatic Tumor Segmentation inpaints diffusion-generated tumors into healthy CT and reports large gains. The same paper reports that the segmenter is extremely sensitive to what happens at the synthetic boundary, where elastic boundary noise drops U-Net Dice from 48.48% to 32.49%, and it notes artifacts in the transition zone between synthetic tumor and real tissue. We read that sensitivity as a hint that the network attends closely to local texture at the lesion edge. Anatomy of Domain Shift Impact on U-Net Layers in MRI Segmentation finds that a U-Net's early encoder layers, the ones whose outputs travel through skip connections straight to the decoder, are the layers most damaged by scanner shift. These are also the layers most able to detect fine texture fingerprints.
One paper supplies the method. Focal Frequency Loss for Generative Models compares generated and real images through their 2D Fourier transforms, including phase, and up-weights the frequencies the generator gets most wrong. It was validated only on natural images and only with paired targets. We reuse it for the paired part of our objective. The genuinely new piece is an unpaired, local spectral-matching term for lesion interiors, where no ground-truth image exists.
Another group of papers constrains the design, because they show that frequency heuristics without a generator already go a long way. Frequency-mixed Single-source Domain Generalization for Medical Image Segmentation (FreeSDG) mixes low-frequency bands across images. FIESTA masks Fourier amplitude while protecting phase. FVP adapts a frozen U-Net with a prompt placed in the low-frequency spectrum. All three treat scanner style as a low-frequency phenomenon and leave high frequencies alone. None of them creates new lesions, and none addresses a cue that is spatially tied to the label. Frequency Domain Adversarial Training for Robust Volumetric Medical Segmentation shows that segmenters collapse under small frequency-space perturbations (UNETR from 74.3% to 32.2% Dice on Synapse). That supports our premise that segmenters are highly sensitive to frequency-coefficient perturbations. Its defense, however, is adversarial training on the segmenter and does not address synthetic data.
Core Claim
The claim has three parts, each falsifiable. First, segmenters trained with inpainted synthetic tumors respond to generator fingerprints by themselves: transplanting only the high-frequency residual of a synthetic lesion into healthy tissue makes them predict "tumor". Second, this sensitivity is concentrated in the shallow encoder levels feeding the skip connections. Third, the shortcut can be removed by changing the generator's training loss, without giving up the gains synthetic tumors provide, and doing so helps more than post-hoc cleanup. If the first part fails, the project reduces to a comparison of frequency-cleaning methods, and we will report it as such.
Research Program
The generator is the latent diffusion pipeline used by Peng et al., a model that creates samples by gradually denoising random noise inside a compressed latent space. A separate autoencoder decodes those latents back to CT voxels. Upsampling decoders are a known source of checkerboard-like high-frequency artifacts, as the Focal Frequency Loss paper documents for natural images, so we fine-tune the autoencoder's decoder. We do not retrain the diffusion model.
The fine-tuning objective has a paired part and an unpaired part. Let $x$ be a real CT patch and $\hat{x}$ its reconstruction through the autoencoder. Let $\tilde{x}$ be a decoded volume containing a synthetic lesion with mask $M$, and let $R$ be a ring of real tissue just outside $M$. The objective is
$$ \mathcal{L} = \|x-\hat{x}\|_1 + \lambda_1 \sum_{s} \mathrm{FFL}_s(x,\hat{x}) + \lambda_2 \sum_{s} \Big\| \log \bar{P}^{\,hf}_s(\tilde{x}; M) - \log \bar{P}^{\,hf}_s(\tilde{x}; R) \Big\|_1 . $$
Here $\mathrm{FFL}_s$ is the focal frequency loss computed on local windows of size $s$. We use several window sizes because one window size cannot resolve both fine and coarse frequencies, the same reasoning behind multi-resolution spectral losses in Parallel waveform synthesis based on generative adversarial networks with voicing-aware conditional discriminators. The term $\bar{P}^{\,hf}_s(\tilde{x}; A)$ is the average high-frequency power spectrum of size-$s$ windows taken from region $A$. The third term therefore asks that the fine texture inside the lesion share the noise and sharpness statistics of the real tissue around it, while leaving phase, and therefore lesion shape and boundary placement, unconstrained. The weights $\lambda_1$ and $\lambda_2$ are set on a held-out validation split.
On the segmentation side, we train a 3D U-Net and SwinUNETR, matching Peng et al., with and without each augmentation. We add two diagnostic instruments. The fingerprint-transplant probe takes healthy pancreas regions and adds only the high-pass residual of synthetic lesions, so no shape or intensity change occurs, then measures how often the segmenter predicts tumor there. The layer probe trains a linear classifier on features from each U-Net encoder level to tell synthetic-lesion patches from real-tumor patches; high accuracy at shallow levels would locate the shortcut in the skip pathway, consistent with the SpotTUnet finding.
Baselines
The naive reference point is a U-Net trained on real MSD-Pancreas data only, with standard spatial and intensity augmentation. This tells us how much any synthetic data helps at all.
The strongest existing method is the unmodified Difftumor augmentation of Peng et al., which reports SwinUNETR rising from 45.88% to 56.10% Dice on MSD-Pancreas when trained with large synthetic tumors. It also reports 48.48% Dice for U-Net in its clean-boundary condition. We rerun it with the same tumor-size schedule, the same number of synthetic cases, the same segmenter training budget, and the same validation-based model selection as our method. Beating it means higher Dice on held-out real tumors, averaged over at least three seeds with non-overlapping confidence intervals, together with a lower fingerprint-transplant firing rate. A Dice tie with a much lower firing rate would still count as a partial positive result, because it means the gains no longer depend on the shortcut.
The second strong competitor is post-hoc recalibration with FreRec applied to Difftumor outputs. FreRec was built for 2D whole-image classification, so we apply it slice-wise to the full volume and also to the lesion region alone, and we report both variants. Its denoising autoencoder gets the same real training data and comparable compute to our decoder fine-tuning. No reported segmentation result was found on Lacuna for FreRec; its reported figure is a classification gain from 0.794 to 0.855 AUC on brain-tumor MRI.
The cheap competitor is fingerprint drowning: low-pass filtering or adding Gaussian noise to synthetic lesions, either inside the lesion only or over the whole volume, with the filter strength tuned on validation. A fourth baseline is FIESTA-style Fourier amplitude augmentation applied on top of Difftumor, which tests whether generic frequency invariance in the segmenter already neutralizes the cue.
Three ablations isolate our ingredients. The first keeps only the paired multi-resolution frequency term. The second keeps only the unpaired lesion-versus-ring term. The third fine-tunes the decoder with the spatial $L_1$ term alone, for the same number of steps, to separate the effect of frequency supervision from the effect of extra decoder training.
Why not just…?
Why not just run FreRec after generation? This is a real threat, and we test it head to head. Our reason to expect a gap is structural. FreRec retrieves a similar real image and swaps its high-frequency content, which fits a fully synthetic image. In an inpainted volume, the surrounding tissue is already real, and the defect is a local mismatch at a lesion boundary to which the segmenter is known to be hypersensitive (Peng et al.'s 48.48% to 32.49% drop). Whole-image replacement can also introduce its own boundary inconsistencies. If FreRec matches our Dice and transplant-probe numbers, we will conclude that the in-training term adds nothing beyond convenience, and we will say so.
Why not just blur or add noise to the synthetic lesions? If applied only inside lesions, blurring or noise replaces one label-correlated cue with another: "smooth or noisy patch means tumor". The transplant probe, rerun with blur residuals, would show this. If applied to the whole volume, it erases fine boundary detail in real tumors too, which the Peng et al. sensitivity result suggests is costly, and it does not match texture to the local tissue. Both variants are run as tuned baselines. If whole-volume noise performs as well as our method, that cheaper fix wins.
Why not skip the generator and use Fourier augmentation (FreeSDG or FIESTA)? These methods randomize style but create no new tumors, while the gain reported by Peng et al. comes from new lesion examples. They also deliberately leave high frequencies intact, which is exactly where the fingerprint lives. We run FIESTA-style augmentation both alone and stacked on Difftumor, so the experiments show whether generic frequency invariance already absorbs the shortcut.
Why not switch to a transformer? The UNETR collapse under frequency perturbations reported in the VAFT paper argues that attention does not confer spectral robustness. In addition, AMOS reports 3D-UNet ahead of Swin-UNetr on CT (89.04% versus 86.32% mean Dice). We nonetheless run every condition on both a U-Net and SwinUNETR.
Proposed Experiments
The primary benchmark is MSD-Pancreas, with real tumors held out for testing and Dice and lesion-level sensitivity as metrics. The decisive mechanistic check is the fingerprint-transplant probe. The baseline Difftumor segmenter should fire on fingerprint-only residuals well above the rate seen for residuals cut from real tumors, and our segmenter should not. The layer probe should show synthetic-versus-real separability concentrated in the shallow encoder levels for the baseline and reduced after our fix.
The domain-shift check uses AMOS, which contains CT from two centers and eight scanner brands and reports drops of over 20% on unseen MRI scanners. We run the pancreas-tumor segmenters on AMOS pancreas regions and count tumor predictions above a minimum volume as false positives, stratified by scanner. AMOS has no tumor labels, so a random sample of flagged cases will be audited for true pathology. The shortcut hypothesis predicts that baseline false positives cluster on scanners whose native high-frequency spectra are closest to the generator fingerprint, measured with the same windowed spectra used in our loss, and that our method flattens this dependence. No reported result was found on Lacuna for tumor false-positive rates on AMOS, so this arm establishes its own reference with the real-only model.
Spectral fidelity of the synthetic data is tracked with the lesion-versus-ring spectral distance and with a slice-level real-versus-synthetic classifier. FID-style scores are reported only as a secondary measure.
The project fails cleanly if the baseline shows no transplant sensitivity. In that case, any Dice gain we observe must come from generally better synthetic texture rather than from removing a shortcut, and the paper's claim narrows accordingly.
References
- Rethinking Bias in Generative Data Augmentation for Medical AI: a Frequency Recalibration Method
- Optimizing Synthetic Data for Enhanced Pancreatic Tumor Segmentation
- Anatomy of Domain Shift Impact on U-Net Layers in MRI Segmentation
- Focal Frequency Loss for Generative Models
- Frequency-Mixed Single-Source Domain Generalization for Medical Image Segmentation
- FIESTA: Fourier-Based Semantic Augmentation With Uncertainty Guidance for Enhanced Domain Generalizability in Medical Image Segmentation
- FVP: Fourier Visual Prompting for Source-Free Unsupervised Domain Adaptation of Medical Image Segmentation
- Frequency Domain Adversarial Training for Robust Volumetric Medical Segmentation
- Parallel Waveform Synthesis Based on Generative Adversarial Networks with Voicing-Aware Conditional Discriminators
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation