Segmentation networks trained on one kind of medical scan, such as cardiac CT, often fail on another, such as cardiac MRI. Much of that failure is concentrated at organ boundaries. CT shows the heart wall as a sharp intensity step. MRI shows it as a soft, noisy ramp. A network that learned what "edge" means from CT has, in effect, learned a contrast threshold that is wrong for MRI.
Anisotropic diffusion is a classical image-processing tool that smooths an image along edges but not across them. It makes the same kind of decision through an explicit, inspectable knob: a threshold that sets how strong a gradient must be to count as a boundary. This note proposes to make that knob self-calibrating. The threshold is measured against the local noise level, so it stops depending on which scanner produced the image. A coarse view of the whole image then decides where a boundary is anatomically expected, and only there is cross-boundary smoothing switched off.
The claim is narrow and testable. A feature-space diffusion layer whose edge test is normalized by locally estimated noise, and whose stopping strength is set by global anatomical context rather than by mixing global features into local ones, will cut boundary errors when transferring CT to MRI with no target-domain data at all. The gains should concentrate exactly on the low-contrast boundary segments where current methods wander.
Why this, and why now
The four directions in the file meet at one weak point. Cross-modality transfer is asymmetric. Structure-Driven Unsupervised Domain Adaptation for Cross-Modality Cardiac Segmentation, which feeds 3D anatomical landmarks and Canny edges into an adversarially aligned segmenter on the Multi-Modality Whole Heart Segmentation benchmark (MM-WHS), reports 85.8% mean Dice for MRI→CT but only 74.8% for CT→MRI. Without adaptation the two directions reach 64.9% and 38.1%. The MM-WHS challenge report, Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation, attributes MRI's difficulty to lower signal-to-noise and fuzzy myocardium boundaries.
Together, these papers motivate the proposal. The hard direction is hard because its boundaries are low-contrast. Its edge cues come from a fixed-threshold detector that was never calibrated for that contrast.
A second motivating result comes from Unsupervised Domain Adaptation via Similarity-based Prototypes for Cross-Modality Segmentation. That method aligns class-averaged features across modalities. It improves MRI→CT Dice to 77.5%, yet its average surface distance (ASD) of 5.1 is worse than plain entropy minimization at 4.6. Averaging features over whole organs improves overlap while leaving perimeters coarse.
Diffusion has already been put inside networks, which supplies the method rather than the gap. P-Mamba places a wavelet-domain Perona–Malik diffusion block, the classical gradient-thresholded smoother, beside a linear-time state-space global branch. A per-pixel gate mixes the two, and the model reaches 0.9221 Dice on pediatric echocardiography. SAR-to-Optical Image Translation via Neural Partial Differential Equations uses a differentiable Perona–Malik module with a PDE loss for radar-to-optical image translation. Semantic Diffusion Network approximates anisotropic diffusion in a single pass with a semantic difference convolution and gains +4.3 boundary F-score on Cityscapes.
None of these three tests whether the diffusion transfers across imaging modalities. That question is the one that matters. In all three, the edge test depends on raw feature or intensity differences, whose scale is precisely what changes between modalities.
Core Claim
The obvious combination would bolt a Perona–Malik block, or the semantic difference convolution, into a CT→MRI adaptation pipeline. This proposal differs in two specific ways, and each is ablated.
The first difference is noise calibration. The conduction function receives the gradient magnitude divided by a robust local noise estimate. It never sees the raw gradient. This makes the edge decision invariant to the intensity gain and noise-level differences between modalities, at least to first order.
The second difference concerns the global branch. It never adds its features to the local stream. Its only output is a low-resolution map of the stopping strength. This settles the local–global trade-off by division of labor. The global branch says where an anatomical boundary should be, and the local diffusion says exactly where and in which direction it runs. The design avoids the feature-blending mismatch that forced CKD-TransBTS to add a dedicated calibration block between its transformer and convolutional features.
Concretely, the layer acts on a decoder feature map $u$ for a small number $T$ of explicit steps:
$$u_{t+1} = u_t + \tau\, \nabla\cdot\big(D\,\nabla u_t\big), \qquad D = g_n\, e_n e_n^\top + g_t\, e_t e_t^\top .$$
Here $\tau$ is the step size. The unit vectors $e_n$ and $e_t$ point across and along the local structure; they are the eigenvectors of a structure tensor, which summarizes local gradient orientation. The tensor is computed on a contrast-normalized guidance map, namely a local rank transform of the input image.
The tangential conduction $g_t$ is a learned constant near one. The normal conduction is
$$g_n = \exp\!\Big(-\big(|\nabla u| \,/\, (\kappa(x)\,\hat\sigma(x))\big)^2\Big),$$
where $\hat\sigma(x)$ is the median absolute deviation of a high-pass response in a local window, and $\kappa(x)$ is the stopping strength predicted at coarse resolution by the global branch. The global branch can be a small attention module or a state-space module. It is cheap, because it outputs a single channel.
Dividing by $\hat\sigma$ is what gives the invariance claim. If a new scanner multiplies the signal by $a$ and its noise scales with it, both $|\nabla u|$ and $\hat\sigma$ scale by $a$, so $g_n$ is unchanged. Real CT→MRI shift is not a pure rescaling, so this is a hypothesis to test, not a guarantee.
Research Program
The work proceeds in three stages. The first stage builds the layer and verifies its invariance in isolation. Source-domain CT slices undergo synthetic shifts: gain, gamma, additive Rician-like noise, and bias field. On these shifted slices we measure how stable the conduction maps $g_n$ are compared with an uncalibrated Perona–Malik block and the semantic difference convolution. If the calibrated maps are not markedly more stable, the central mechanism fails, and we stop there.
The second stage, the main study, is domain generalization: train on MM-WHS CT, test on MM-WHS MRI with no MRI images seen. This is stricter than the unsupervised adaptation setting used by prior work. It isolates whether the operator itself carries the transfer.
The third stage inserts the same layer into an adaptation pipeline modeled on Structure-Driven UDA. The Canny-edge input is replaced by the calibrated diffusion output. The question is whether the hard-direction gap narrows.
A secondary transfer test uses retinal vessels, from fundus photography to OCTA-500. Vessels are thin, low-contrast tangential structures, which is the case orientation steering is meant to protect. In-domain sanity checks on ACDC cine-MRI and EchoNet-Dynamic confirm that the layer does not cost accuracy when there is no shift.
The analysis that makes the result interpretable is stratification by boundary contrast. For every ground-truth boundary point we compute a local contrast-to-noise ratio, then report surface error within contrast bins. The proposal predicts the largest gains in the lowest bins. If gains are uniform across bins, the layer is acting as generic regularization, and the boundary story is wrong.
Baselines
The naive reference is a 2D U-Net trained on CT and applied directly to MRI. Prior work reports this "without adaptation" lower bound at 38.1% mean Dice for CT→MRI and 64.9% for MRI→CT (Structure-Driven UDA). We retrain it ourselves with the identical backbone, so that every other arm differs only by the added component.
The strongest generalization competitor we can construct is the same U-Net trained with aggressive intensity augmentation: random gamma, gain, noise, bias field, and blur. This is the most plausible way to learn contrast invariance without any operator. All proposed arms are trained with this same augmentation, so that any gain is over and above it. No reported result was found on Lacuna for intensity-augmentation generalization on MM-WHS CT→MRI.
The strongest existing method on the hard direction is Structure-Driven UDA, at 74.8% mean Dice CT→MRI and 85.8% MRI→CT. Because it uses unlabeled target images, we compare it against our stage-three variant, which uses the same target access. Our target-free stage-two model is reported alongside it only as a gap measurement, never as a head-to-head comparison. For surface distance, the reference point is ASD 5.1 for prototype alignment versus 4.6 for entropy minimization on MRI→CT (Prototype UDA). On retinal transfer, the reference is AdaptDiff, a conditional generative model that synthesizes target-style training images. It reaches 81.94% Dice on OCTA-500 against a 73.16% source-only baseline and an 84.78% supervised oracle (AdaptDiff).
Operator baselines isolate the key ingredient. One arm uses a scalar, uncalibrated Perona–Malik block with a learned global threshold, in the style of P-Mamba's diffusion branch, within our backbone. A second arm uses the semantic difference convolution neck from Semantic Diffusion Network. A third ablation removes noise normalization but keeps orientation and global gating. A fourth keeps the normalization but fixes $\kappa$ to a constant. A fifth replaces the gating-only global branch with standard additive fusion.
All arms share data splits, epochs, optimizer, and a matched hyperparameter search of equal trial count. Model selection uses source-domain validation only; no target labels are ever used for tuning. That rule matters, because generalization results tuned on target labels are inflated.
Beating the field means two things. First, the target-free model must exceed the augmented U-Net on CT→MRI Dice and ASD with confidence intervals over three seeds. Second, the stage-three model must exceed 74.8% CT→MRI Dice. A further qualitative condition applies: the advantage over the uncalibrated Perona–Malik arm must be concentrated in the low-contrast boundary bins.
Why not just…?
Why not just use heavy intensity augmentation? This is the real threat. Augmentation can teach contrast invariance implicitly, and if it does so fully, the operator adds nothing. We do not argue it away. The augmented U-Net is the main baseline, and all our arms also use augmentation. If the calibrated layer gives no gain over augmentation alone, we will report that synthetic intensity perturbation suffices for this shift, and the contribution reduces to the invariance diagnostic.
There is a reason to expect a residual gain. Augmentations perturb global intensity but rarely reproduce MRI's spatially varying noise and partial-volume blur at specific boundaries. A per-pixel noise estimate tracks exactly that variation.
Why not just pre-filter the input images with classical anisotropic diffusion? A fixed classical filter is cheaper and needs no training. However, its threshold must be set per modality, which is the very miscalibration at issue, and the file documents the sensitivity of classical diffusion to its conduction parameter. We include an arm with input-level Perona–Malik preprocessing, its threshold tuned on CT, applied unchanged to MRI. We also include the same arm with noise-normalized thresholding. The second arm tests whether calibration alone, without learning or global gating, captures most of the benefit. If it does, the simpler preprocessing pipeline is the right recommendation.
Why not just add a boundary or contour loss? Contour branches supervised by Canny edges of the masks, as in the contour-loss U-Net, improve in-domain surface distance; for example, they lower spleen CT HD95 from 7.5744 to 5.8814. A loss, though, teaches where CT boundaries are. It does not change how the network detects a boundary in an unseen contrast regime. We run this arm under the same transfer protocol. A boundary loss that helps in-domain but not in the low-contrast MRI bins would support our account.
Why not just use the existing semantic difference convolution? It already performs one-pass anisotropic diffusion with good boundary gains. Its edge test, however, is built from raw feature differences and semantic similarity computed on features that themselves shift across modalities. It is a direct arm in our comparison. Our stage-one conduction-stability test is designed to show whether its maps drift under synthetic intensity shift while the calibrated maps do not.
Decisive Checks
Three outcomes would decide the question. First, under synthetic gain and noise shifts, the calibrated conduction maps must stay stable where the uncalibrated ones drift. Second, CT→MRI surface-error reductions must be concentrated in the lowest contrast-to-noise boundary bins and must survive the augmentation baseline. Third, inserting the layer into a Structure-Driven-style adaptation pipeline must move CT→MRI mean Dice above the reported 74.8%, with MRI→CT surface distance improving on the reported 4.6–5.1 range. If the first check fails, the mechanism is wrong. If the second fails, the layer is generic regularization. If only the third fails, the operator is a generalization tool that does not compound with adaptation. Each outcome is worth reporting.
References
- Structure-Driven Unsupervised Domain Adaptation for Cross-Modality Cardiac Segmentation
- Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge
- Unsupervised Domain Adaptation via Similarity-based Prototypes for Cross-Modality Segmentation
- P-Mamba: Marrying Perona Malik Diffusion with Mamba for Efficient Pediatric Echocardiographic Left Ventricular Segmentation
- SAR-to-Optical Image Translation via Neural Partial Differential Equations
- Semantic Diffusion Network for Semantic Segmentation
- CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer With Modality-Correlated Cross-Attention for Brain Tumor Segmentation
- AdaptDiff: Cross-Modality Domain Adaptation via Weak Conditional Semantic Diffusion for Retinal Vessel Segmentation
- A Medical Image Segmentation Method based on Multi-scale Features and Contour Loss Constrain (S)