Equalize, Then Generate: Sharper Synthetic Tumors from an Invertible 3D Contrast Space

Diffusion models, which are generators that learn to turn random noise into realistic samples by removing noise step by step, can paste synthetic tumors into healthy CT scans. Segmentation networks trained on these scans improve noticeably. They are also very sensitive to how precisely the synthetic tumor's edge matches its label. On pancreatic tumors, which look almost the same as the surrounding tissue, slightly blurred or misaligned boundaries in the training data drop downstream accuracy by about a third. We think a large share of this blur is introduced at a specific point. Before diffusion runs, a compression network squeezes each scan into a compact latent code. That network is trained on globally normalized intensities, and a margin separated from its surroundings by only a few Hounsfield units (HU, the physical intensity scale of CT) contributes almost nothing to its reconstruction error, so it learns to smear those margins.

We propose to train the compression network and the diffusion model in a locally contrast-equalized space. A 3D version of contrast-limited adaptive histogram equalization (CLAHE) stretches small local intensity differences until they are large enough to matter. Our 3D construction is exactly invertible voxel by voxel, so generated tumors can be mapped back to true HU values afterwards. A second, cheaper mechanism handles the blur that remains. Each synthetic voxel gets a reliability weight based on how much the generator itself disagrees about the boundary location at that voxel, and the segmenter's training loss is scaled by that weight. This treats the generator as an unreliable sensor whose contribution is turned down where it is untrustworthy. The test is whether these two changes beat the strongest reported synthetic-augmentation result on MSD-Pancreas. A second test is whether they explain away the odd finding that large synthetic tumors help more than realistic small ones.

Core Claim

The bottleneck for synthetic tumor augmentation in low-contrast organs is boundary fidelity, and much of the boundary error is created by the latent autoencoder, not by the diffusion process itself. Equalizing intensities locally in 3D before encoding, then inverting after decoding, should produce sharper tumor margins at correct physical intensities. That should translate into higher Dice and lower boundary distance on real test scans. Down-weighting the remaining unreliable boundary voxels during segmenter training should add a further, smaller gain.

The motivating evidence comes from Optimizing Synthetic Data / Difftumor. That work synthesizes pancreatic tumors with a 3D latent diffusion model and shows two things. First, SwinUNETR rises from 45.88% to 56.10% Dice with large synthetic tumors. Second, U-Net collapses from 48.48% to 32.49% when elastic noise is added to the label boundaries. The second result tells us the boundary is where synthetic data succeeds or fails. The first tells us that the tumor distribution that works best is not the realistic one. Our hypothesis connects the two: large tumors may help mainly because their margins survive autoencoding better, since more of their volume lies far from the blurred edge. If that is right, sharper small tumors should close much of the gap.

How the Pieces Fit, and What Is New

Two volumetric generators define the starting point. Medical Diffusion compresses 3D volumes with a 3D VQ-GAN, a compression network trained with an adversarial sharpness loss, before running latent diffusion. MedGen3D generates paired masks and textures in stages. Both supply our generative method, and both train on raw or globally normalized intensities. We keep their architecture and change only the intensity space in which it operates. Earlier lesion-injection work with conditional GANs, such as 3D multi-conditional nodule placement and synthetic lung nodules for segmentation robustness, likewise operates on unequalized volumes.

Local contrast normalization has paired with generative augmentation only in 2D. Deng and Yaermaimaiti apply CLAHE to electroluminescence images before training a DCGAN on solar-cell microcracks. Similar 2D gains are reported for retinal non-perfusion areas in NPA-Net and for faster training on chest X-rays in Chen and Ruan. These works motivate the idea that equalization exposes structure a learner would otherwise miss. They also constrain our design in two ways. Their equalization is one-way, which distorts physical units. And applying it slice by slice to a 3D scan causes intensity banding between slices. Our contribution is a 3D, invertible equalization used as a reversible coordinate change for generation, not as a one-way preprocessing step.

The reliability weighting borrows its logic from multi-sensor fusion. In Guan et al., MVDNet, and RoboFusion, attention-style gates suppress a corrupted sensor, such as LiDAR in fog, so that it cannot pollute clean streams. Those systems learn their gates from paired real data that contains many kinds of corruption. That data does not exist for synthetic CT. Our weight is therefore computed from the generator's own uncertainty, not learned by attention.

Research Program

The invertible 3D equalization. We first clip each CT volume to an abdominal intensity window. We then divide it into 3D tiles and compute a clipped, redistributed histogram per tile, as in standard CLAHE. Redistribution spreads the clipped mass uniformly over all bins, so every bin ends up with positive mass. Each tile's mapping $F_k$ is therefore strictly increasing. The mapping at voxel $p$ blends the eight surrounding tile mappings:

$$T_p(v) = \sum_{k} w_k(p)\, F_k(v),$$

where $v$ is the voxel's intensity and $w_k(p) \ge 0$ are trilinear weights that sum to one. A non-negative blend of strictly increasing functions is strictly increasing. So $T_p$ can be inverted at every voxel by one-dimensional bisection, given the stored tile mappings, which take a few kilobytes per scan. Blending in 3D, not 2D, removes slice banding by construction.

When a tumor is generated inside a healthy host scan, we apply and later invert the host's own $T$. That transform is fixed and does not depend on the generated content, so the round trip returns exact HU values up to floating-point error. Tile size and clip limit become two hyperparameters, and we sweep them. A clip limit near zero recovers ordinary global normalization, which gives a built-in ablation.

Generation. We retrain the Difftumor-style pipeline, meaning the 3D autoencoder plus the latent diffusion model, in the equalized space and map its outputs back to HU. Everything else stays identical to the raw-space version: architecture, latent compression factor, number of training steps, and tumor mask distribution. We measure three things directly. The first is autoencoder reconstruction error in a three-voxel band around real tumor margins, reported in HU after inversion. The second is the intensity gradient across synthetic versus real tumor boundaries. The third is the distance between the HU histograms of synthetic and real tumor interiors.

Reliability weighting. For each synthetic mask we draw $K$ diffusion samples, with $K$ around four, computed offline. We estimate the boundary each sample actually produces by thresholding its contrast relative to the host. The per-voxel reliability $r(p)$ is the fraction of samples whose produced boundary agrees with the label at $p$. The segmenter's loss on synthetic voxels is multiplied by $r(p)^{\gamma}$, while real voxels and voxels far from any boundary keep weight one. Here $\gamma$ is a single exponent we tune. This is the only part of the system that is dynamic. It turns the generator down exactly where the generator is unsure, and it costs nothing at test time.

Baselines

The naive reference point is a segmenter trained on real MSD-Pancreas data alone with standard spatial and intensity augmentation. Difftumor reports this as 45.88% Dice for SwinUNETR and 48.48% for U-Net (Peng et al.). We reproduce these numbers on the same splits before claiming anything, and we use nnU-Net-style training as the backbone recipe so the baseline is not artificially weak.

The strongest existing method is Difftumor's raw-space latent diffusion augmentation with its best "large tumor" setting, reported at 56.10% Dice with SwinUNETR. We rerun it ourselves with the same number of synthetic volumes, the same host scans, the same segmenter schedule, and the same budget for tuning hyperparameters. Five seeds per configuration give variability. Beating it means a mean Dice gain of at least two points on real test tumors, with seed intervals that do not overlap, together with a reduction in 95th-percentile Hausdorff distance. A Dice gain alone could come from better detection of tumor cores instead of better edges, so the boundary metric is required.

A second strong baseline is CLAHE applied only to the segmenter's input, with the generator left in raw space. This tests whether the gain comes from equalized generation or simply from the segmenter seeing equalized images.

Two ablations isolate our ingredients. The first is equalized generation without reliability weighting. The second is raw-space generation with reliability weighting. A third ablation replaces our 3D equalization with slice-wise 2D CLAHE, which measures what volumetric consistency contributes.

Why not just…?

Why not just apply CLAHE to the segmenter's inputs and leave the generator alone? This is the cheapest version of the idea. It is a genuine possibility if segmenters, not generators, are the ones failing to see faint margins. It cannot, however, repair a margin the autoencoder has already smeared, because equalizing a blurred edge sharpens noise, not the edge. The input-CLAHE baseline settles the question directly. If it matches equalized generation, our central claim is wrong. The band-reconstruction measurement would then show whether the autoencoder was ever the bottleneck.

Why not just add a boundary-weighted reconstruction loss to the autoencoder? This is the most serious threat. Equalization partly acts as a reweighting of reconstruction error in intensity space, and an edge- or mask-weighted loss is simpler. The two differ in one respect: equalization also enlarges small intensity gaps relative to the diffusion model's noise at every denoising step, which a reweighted autoencoder loss does not do. We run the weighted-loss autoencoder under the same compute budget. If it matches our boundary metrics and downstream Dice, we will report that the simpler fix suffices and drop equalization from the recommendation.

Why not just down-weight all synthetic data by a fixed factor, or filter out bad samples? A single global weight cannot tell a sharp synthetic interior from a dubious synthetic edge. Difftumor's label-noise collapse suggests the damage is concentrated at edges. Filtering whole samples throws away good interiors along with bad edges. We include both a tuned fixed weight and discriminator-based sample filtering as baselines for the reliability term. Label rectification of the kind performed by DiffRect is another option, which corrects labels instead of weighting them. We treat it as a stretch comparison because it targets semi-supervised pseudo-labels, not synthetic images.

Why not just use classical elastic and intensity augmentation? Geometric transforms of the kind used in 3D U-Net create no new tumor appearances. The roughly ten-point Difftumor gain over the real-only baseline is already evidence that synthesis adds signal on this task. The naive baseline keeps this comparison honest.

Proposed Experiments

All main experiments use MSD-Pancreas with SwinUNETR and U-Net, matching Difftumor so that the reported 56.10% and 48.48% numbers can anchor our results. The primary outcome is Dice on real test tumors. Results are stratified by tumor diameter, because small, low-contrast tumors are where we expect the method to help and where clinical value lies. Secondary outcomes are 95th-percentile Hausdorff distance and HU fidelity of synthetic tumors.

The first decisive check concerns the autoencoder itself. Equalized generation must reduce boundary-band reconstruction error in HU, after inversion, relative to raw-space training. If it does not, the downstream experiments are uninformative and the project stops there.

The second decisive check is the size tension. We rerun Difftumor's sweep over synthetic tumor sizes in both intensity spaces. Our hypothesis predicts that the advantage of large over small synthetic tumors shrinks substantially in equalized space. If the large-tumor advantage persists unchanged, the size effect has some other cause, perhaps acting as a curriculum, and our explanation is falsified even if overall Dice improves.

The third check is quantitative integrity. Round-trip error must be negligible, and the HU histogram of healthy tissue in augmented scans must be unchanged. Otherwise equalization has leaked into the physical units it promises to preserve.

The fourth check tests the reliability weight. We deliberately inject Difftumor's elastic label noise into synthetic masks, then ask whether reliability weighting recovers a meaningful part of the documented 48.48% to 32.49% collapse, which a fixed global weight should not. A strong result would be recovering most of that drop. A weak result would mean the generator's own disagreement is a poor proxy for label error.

The fifth check is volumetric consistency. Comparing slice-wise against 3D equalization should show measurable banding along the depth axis in the slice-wise synthetic volumes, and lower boundary accuracy in through-plane directions.

If budget allows, we repeat the generation-side comparison on LIDC-IDRI lung nodules, a dataset Medical Diffusion already uses. This shows whether the effect is specific to the pancreas. No reported downstream segmentation result was found on Lacuna for synthetic-nodule augmentation on LIDC-IDRI, so that experiment would establish its own baselines.