A segmentation network trained on CT scans usually fails on MRI. The anatomy is the same, but the mapping from tissue to gray value is different, and it is often reversed: a boundary that is bright-to-dark in one modality can be dark-to-bright in the other. We propose to remove this source of failure at the input instead of hoping the network learns to ignore it. At every voxel, we take a translation-invariant wavelet transform, meaning a multiscale filter bank that does not downsample, so every scale stays aligned to the original voxel grid. We collect the local detail coefficients across scales and orientations into a vector and keep only its direction. Wavelet filters have zero mean, so any intensity remapping that is roughly affine over a neighborhood multiplies this vector by a scalar, possibly a negative one. The direction is therefore unchanged up to sign. A segmenter trained on these signatures from a single source modality should see nearly the same input on an unseen modality wherever the boundary is visible at all.
This matters because the strongest current single-source methods buy contrast robustness with expensive generative models that can invent anatomy. Fine-tuned diffusion models produce synthetic target-style training images, and the authors themselves report blurred or hallucinated fine structures such as the posterior vertebral arch. Our representation needs no generator and no target data. It also makes a falsifiable prediction about where it will fail: at boundaries whose contrast vanishes in the target modality, and in textured regions where the local-affine assumption breaks.
Core Claim
Let $I$ be a source image and $I'$ the same anatomy imaged in another modality. Within a neighborhood of voxel $x$ whose size matches the coarsest wavelet support, suppose $I'(y) \approx a(x)\,I(y) + b(x)$. The undecimated detail coefficients $d_{j,o}(x) = (\psi_{j,o} * I)(x)$, at scale $j \le J$ and orientation $o$, then satisfy $d'(x) \approx a(x)\,d(x)$, because each wavelet $\psi_{j,o}$ integrates to zero and cancels $b(x)$. The signature we feed the network is
$$u(x) = s(x)\,\frac{d(x)}{\lVert d(x) \rVert}, \qquad c(x) = \frac{\lVert d(x)\rVert}{\lVert d(x)\rVert + \tau\,\hat\sigma(x)},$$
where $s(x)\in\{\pm1\}$ fixes the sign so that the largest-magnitude component of $d(x)$ is positive. The gate $c(x)$ down-weights voxels whose detail energy is near the locally estimated noise level $\hat\sigma(x)$, and $\tau$ is a threshold. Under the local-affine model, $u(x)$ is exactly invariant to $a(x)$ and $b(x)$, including contrast inversion. Across scales, $u(x)$ also tells apart a step edge, a thin line such as a vessel or cortical rim, and a blob, which a single-scale gradient cannot.
Translation invariance is what makes this a per-voxel statement. In the decimated transform used by cWDM, which runs 3D diffusion on standard wavelet subbands, the coefficient at scale $j$ exists only on a $2^j$ grid. The vector assembled at a voxel therefore mixes coefficients from different positions depending on grid phase. Wang et al. show directly that a one-sample shift scatters decimated-transform energy across subbands, worst at high frequencies, while the redundant transform shifts cleanly. A cross-scale signature built on decimated coefficients would change when the anatomy moves by one voxel, which is exactly the kind of nuisance a cross-modality model sees all the time. The new element is not "use wavelets for domain generalization". It is the specific invariant: a per-voxel, sign- and scale-normalized, multiscale coefficient direction that is exact only under an undecimated transform.
Relation to Prior Work
Three lines of work motivate the gap. Zhou et al. treat modality shift as a gray-level remapping. They simulate it with random Bézier curves and route test images through one of two batch-normalization paths. Chen et al. model scans as independent content and style factors, synthesize target styles with a mask-conditioned ControlNet on a DreamBooth-tuned Stable Diffusion, and add a patch-wise contrastive loss. Both teach invariance through data. We build it into the input, and we test whether that makes the expensive synthesis unnecessary.
QUAVE is the closest wavelet-side work. It feeds quaternion-wavelet subbands, which are only approximately shift-invariant, into quaternion networks on 2D slices, and keeps 4 of 16 subbands chosen by a super-resolution pretext task. It does not normalize away contrast, it is not exact under shifts, and it is not volumetric. We differ on all three counts.
Two papers supply methods we reuse. SMASH by Xing et al. combines translation-invariant wavelets with empirical-Bayes shrinkage under spatially varying noise. We borrow its idea of estimating a spatially varying variance per scale for $\hat\sigma(x)$. CT, MRI and X-ray have very different and non-uniform noise, so a global threshold would gate the wrong voxels.
Ram et al. constrain the design. They show that building redundancy into the transform avoids the cost of cycle spinning, which shifts the signal over all offsets and averages. The storage problem nonetheless remains: an undecimated 3D transform over $J$ levels stores $(7J+1)$ volumes. Our signature keeps only the normalized detail direction, 7J channels plus one gate. With $J=3$ that is 22 input channels, small next to the backbone's own feature maps.
Research Program
We will build a 3D undecimated Haar and Daubechies-4 filter bank on the GPU with dilated filters, the à trous scheme. We will compute $u$ and $c$ on the fly and feed them, concatenated with the intensity image after random Bézier augmentation, to a standard 3D U-Net. Keeping the intensity channel is deliberate. Some boundaries are invisible to the signature, for example where two tissues share an intensity in the source, and the intensity channel lets the network use whatever is truly stable, such as air versus body.
The first analysis tests the invariance itself rather than downstream Dice. BraTS provides co-registered T1, T1ce, T2 and FLAIR volumes of the same patient. For matched voxels across sequences we will measure the cosine agreement of $u$ as a function of the gate $c$. We will compare it with the agreement of raw intensities, raw wavelet coefficients, single-scale normalized gradients, the decimated-transform signature, and features from an intensity-augmented network.
The second analysis maps where the assumption fails. Using the BraTS segmentation labels, we will split boundaries into tissue-pair types and report signature agreement per type. The theory predicts high agreement at two-tissue edges with nonzero contrast in both sequences and low agreement at junctions and inside tumor texture.
Baselines
The naive reference point is DeepAll: the same 3D U-Net trained on the source modality with standard spatial and intensity augmentation. In Zhou et al., the dual-normalization method reaches 54.44% Dice on the BraTS T2-source cross-sequence benchmark, reported as a 19.27% improvement over DeepAll. The lumbar spine benchmark used by Chen et al. lists no reported Dice figure on Lacuna.
The strongest heuristic competitor is Zhou et al.'s Bézier augmentation with dual normalization. It is the cheapest existing method aimed at exactly the gray-level remapping our invariant targets. We will rerun it with the same backbone, the same epochs, and the same hyperparameter search budget, about 20 configurations per method. Beating it means higher mean Dice on every unseen target modality across three seeds, with non-overlapping bootstrap intervals.
The strongest existing method is the causal diffusion intervention of Chen et al.. It reports 86.20% average Dice for CT to MRI on AMOS, 1.13 points from the in-domain oracle, and raises CT to X-ray lung Dice from 67.33% to 78.79%. Because it is so close to the oracle on AMOS, we do not expect to beat it there. Success on AMOS means coming within one Dice point at a small fraction of the training compute, which we will report in GPU-hours including diffusion fine-tuning and sampling. Beating it means higher Dice on lung CT to X-ray and on lumbar spine, with specifically better boundary distance on the posterior vertebral elements the authors identify as hallucinated. Where their synthetic data or code cannot be reproduced, we compare against their reported numbers on matching splits and say so.
Four ablations isolate the key ingredient. The first replaces the undecimated transform with the decimated one, upsampled to voxel grid. The second uses cycle-spun decimated coefficients. The third drops normalization and feeds raw undecimated coefficients. The fourth collapses to one scale, giving a normalized gradient. The decimated ablation is the critical one: if it matches the full method, translation invariance is not doing the work, and the claim fails.
Why not just…?
Why not just augment intensities more aggressively and let the network learn invariance? This is the main threat. Bézier augmentation as in Zhou et al. applies one global monotone or inverting curve per image. Real modality shifts remap tissue pairs differently at different places, so a boundary flips sign in one region and not another. A global curve cannot produce that, while our invariant handles any local sign. The paired-BraTS agreement test settles this directly. If an intensity-augmented network's intermediate features are already as consistent across sequences as $u$, and Dice matches, the wavelet signature adds nothing and we will report that.
Why not just use a normalized image gradient or local contrast normalization, which is also invariant to local affine remapping? It is, at one scale. A single scale cannot tell a thin bright line from a step edge, and it is dominated by noise in low-signal modalities such as X-ray and fast MRI. The single-scale ablation tests this. We expect the gap to be largest on thin structures, such as vertebral processes and cardiac walls in MM-WHS. If the gap is negligible, the simpler gradient input is the right recommendation.
Why not just use the decimated wavelet transform or quaternion wavelets, which are cheaper? Wang et al. show decimated subbands change shape under unit shifts, and QUAVE's quaternion transform is only approximately shift-invariant. Whether that matters for segmentation, as opposed to for signal analysis, is an empirical question. The decimated and cycle-spun ablations answer it, and we will also evaluate after random one-voxel test-time shifts to expose the difference.
Why not just generate target-style data with diffusion, as Chen et al. do? It works well on AMOS, but it needs target-style prompts, heavy compute, and a generator that can blur fine anatomy. Our method needs none of these. The head-to-head on spine and X-ray, plus the compute accounting, decides whether that trade is worth it.
Proposed Experiments
The decisive first check is the paired BraTS invariance measurement. If gated signature agreement across T1, T2 and FLAIR is not clearly higher than raw-coefficient and decimated-signature agreement, we stop before any downstream training.
If it passes, we run single-source generalization on four settings: AMOS CT to MRI, across the 15 organs and with held-out MRI scanners, where the AMOS benchmark reports a drop of over 20% in mean Dice for unseen MRI scanners; the BraTS cross-sequence task of Zhou et al.; lung CT to X-ray and lumbar spine CT to MRI to X-ray from Chen et al.; and MM-WHS MR to CT. On X-ray, where the transform is 2D, the signature has 3J orientation channels. We report Dice, average surface distance, Dice under random one-voxel test shifts, and training compute.
The last check targets predicted failures. We stratify target-domain errors by whether the boundary has measurable contrast in the target. The method's advantage over Bézier augmentation should concentrate on contrast-inverted boundaries and vanish on boundaries that disappear in the target. If instead the gains are uniform or absent, the mechanism is not what we claim.
References
- cWDM: Conditional Wavelet Diffusion Models for Cross-Modality 3D Medical Image Synthesis
- Intelligent Extraction of Salient Feature From Electroencephalogram Using Redundant Discrete Wavelet Transform
- Generalizable Cross-modality Medical Image Segmentation via Style Augmentation and Dual Normalization
- Generalizable Single-Source Cross-Modality Medical Image Segmentation via Invariant Causal Mechanisms
- Generalizing medical image representations via quaternion wavelet networks
- Flexible Signal Denoising via Flexible Empirical Bayes Shrinkage
- Redundant Wavelets on Graphs and High Dimensional Data Clouds
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation