Most CT and MRI scans have far coarser resolution between slices than within a slice, and the gap varies from scanner to scanner, protocol to protocol, and hospital to hospital. Three-dimensional segmentation networks are known to degrade sharply as this slice spacing widens. On LiTS liver tumors, for example, a 3D U-Net reaches 51.0% Dice against 66.3% for an anisotropy-aware mesh network, and it collapses as spacing moves from 1 mm to 5 mm (MNet). The field's responses so far have been architectural. Some methods use factorized kernels, others mix 2D and 3D paths, others add cross-slice attention modules that can reach hundreds of millions of parameters, and others synthesize the missing slices with generative models before segmenting. We propose something cheaper and more diagnostic. We treat the slice axis the way speech recognition treats the time and frequency axes of a spectrogram. During training, we randomly erase contiguous bands of the volume's frequency content along the slice direction, and we also erase contiguous runs of slices, so that an ordinary 3D network cannot depend on any single through-plane frequency band.
The physical argument is what makes this more than borrowing an audio trick. Thick slices act as a low-pass filter along the slice axis, so they remove high through-plane frequencies. Gaps between slices cause undersampling, so energy from high frequencies folds back and contaminates the middle bands. A network trained at one spacing learns to rely on through-plane frequency bands whose contents change at another spacing. Removing a band with zeros mimics blur. Replacing it with realistic random energy, in the style of distribution-preserving mask fills from speech recognition, mimics aliasing. The core claim is that a single plain 3D network trained with mixed band erasure and band corruption along the slice axis will have a much flatter accuracy-versus-spacing curve than the same network trained with physically simulated thick slices. It should do this at negligible cost and without new parameters. A by-product is a through-plane spectral sensitivity profile that shows which slice-axis frequency bands a trained segmenter actually depends on.
What exists and what does not
The anisotropy literature supplies the problem and the competing remedies. Cascaded anisotropic CNNs split 3D kernels into in-plane and through-plane parts. A hybrid 2.5D network uses 2D convolutions until the feature maps become roughly isotropic, and MNet learns weights between parallel 2D and 3D paths. Cross-slice attention methods treat slices as tokens: CAT-Net does so at roughly 398M to 610M parameters, CSAM does so at 36.8M, and CSA-Net restricts attention to slice triplets. All of these change the architecture and train on the spacing distribution they happen to receive. None of them regularizes what the network may rely on along the slice axis, which is the lever we pull. Slice Imputation interpolates intermediate slices the way video methods interpolate frames, which adds a heavy preprocessing stage. We instead make the segmenter tolerant of missing through-plane information. CARVE finds that adjacent slices are highly redundant, so in-plane resolution matters more than slice count at fixed compute. That finding motivates us: if the redundancy exists, a network should be able to learn to exploit it when bands are withheld.
Two lines of work bring frequency or masking ideas into medical segmentation, and we differ from both. SF-UNet applies a 2D Fourier transform to intermediate feature maps and learns a deterministic filter on their low frequencies. It works in-plane, without randomness, and only on 2D or near-isotropic data. Our masking is random, runs along the slice axis, and is applied during training only. AMLP and AnatoMask mask spatial patches for self-supervised pretraining. They do not mask frequency bands, and they do not mask along a single anatomical axis.
The method itself comes from audio. SpecAugment shows that masking contiguous bands works where isolated pixel dropout fails, because convolutions interpolate across scattered holes but not across whole bands. f-SpecAugment shows that masking within the local context window, rather than across the whole utterance, keeps alignment-sensitive targets intact. This matters for us because dense segmentation is alignment-sensitive. AugRepl/AugMult show that filling masks with values drawn from the batch's range beats zero-filling. We reinterpret that result physically: zero-fill corresponds to blur, and plausible-energy fill corresponds to aliasing. The audio literature also constrains the design. Masks that are too wide destroy sparse events, and that risk is acute for small lesions. We therefore cap total masked bandwidth, never mask the zero-frequency (mean) component, and limit runs of masked slices relative to lesion extent.
The masking operation
Take a training patch $x(u,v,z)$, where $u,v$ are in-plane coordinates and $z$ is slice index. Compute its 1D discrete Fourier transform along $z$ alone, $X(u,v,k)$, where $k$ is through-plane frequency. We sample one or two contiguous bands $B=[k_0,k_0+w)$, which may include the high-frequency tail but exclude $k=0$. We then form
$$\tilde X(u,v,k) = \begin{cases} X(u,v,k) & k\notin B \\ \alpha\, X(u,v,k) + (1-\alpha)\, |\hat X_b(k)|\, e^{i\phi(u,v,k)} & k\in B \end{cases}$$
Here $\alpha\in\{0,1\}$ selects between two fill types. When $\alpha=0$, the band is replaced by a random-phase signal $\phi$ whose magnitude $|\hat X_b(k)|$ is the batch-average spectrum at frequency $k$; this is the aliasing-like fill. When $\alpha=1$, the band is left intact; zero-fill, the blur-like option, is a third setting obtained by dropping the fill term. We invert the transform to get the augmented patch. Labels are not transformed, because the anatomy has not moved; the network must recover it from what remains.
Separately, with some probability we replace short contiguous runs of slices with interpolations from their neighbours. This is the analogue of SpecAugment's time masking, and it simulates missing or gapped slices. All masking is applied per patch, which follows the local-window lesson of f-SpecAugment. The extra cost is one FFT along a short axis per patch.
Research program
The program has three parts. First, we build the augmentation as a drop-in addition to the nnU-Net augmentation pipeline, with a plain 3D configuration, so that every comparison isolates the regularizer. Second, we build a measurement protocol for anisotropic resolution sensitivity. We start from the thinnest-slice cases available, generate test versions at nominal spacings from about 1 mm to 5 mm using a realistic slice-profile average followed by subsampling, and evaluate on native thick-slice cases. The resulting accuracy-versus-spacing curve is the primary object of study, rather than a single Dice number. Third, we build the spectral sensitivity profile: at test time we erase each through-plane band in turn and record the loss in Dice. This tells us whether a vanilla network concentrates its reliance on a few bands, and whether our training spreads that reliance out. The mechanistic hypothesis predicts both results; if the vanilla network's reliance is already flat, the motivation is wrong.
Baselines
The naive reference point is 3D nnU-Net with its default augmentation and spacing-driven configuration, trained on the native training spacings. On LiTS tumor, MNet's paper reports nnU-Net at 62.0% Dice and 3D U-Net at 51.0% (MNet). It is the network our augmentation is added to, so any gain is attributable to the augmentation alone.
The strongest existing anisotropy method on our main benchmark is MNet, which reports 66.3% Dice on LiTS tumor (MNet). On the Decathlon brain tumor task, Slice Imputation reports 87.00% Dice against 82.91% for a U-Net (Slice Imputation). We will retrain both under our split and our spacing-sweep protocol, with the same epochs and the same per-method tuning budget, because the published numbers come from different splits. Beating MNet means one of two outcomes. The first is matching its Dice at native spacing while losing significantly less Dice between 1 mm and 5 mm test spacing, measured over at least three seeds with patient-level bootstrap intervals. The second is a plain 3D nnU-Net plus masking closing most of the nnU-Net-to-MNet gap with no added parameters. We will also add the masking on top of MNet to test whether the two effects are complementary.
The baseline most likely to match us is physics-based thick-slice simulation. Here training patches are randomly re-acquired at a sampled slice thickness and gap and then resampled back to the training grid. We found no reported result on Lacuna for this augmentation on these benchmarks, so we will implement it with the same application probability and the same range of spacings as our method.
Two further baselines isolate ingredients. Random slice dropping or Cutout alone at matched masked volume removes the frequency-band component. Our method with zero-fill only and high-tail-only bands removes, respectively, the aliasing-like fill and the mid-band masking. If high-tail zero-fill matches the full method, the method has reduced to blur simulation and the band and fill ideas add nothing.
Why not just…?
Why not just resample everything to isotropic voxels before a 3D network? Interpolation adds blur and stair-step artifacts and inflates memory without adding information. On the brain tumor task, linear interpolation reached 68.69% Dice against 82.91% for a plain U-Net (Slice Imputation). CSA-Net also attributes Swin UNetR's weaker infant-brain results to interpolation artifacts (CSA-Net). We include it in the sweep only as a reference.
Why not just simulate thick slices physically? This is the real threat. Thick-slice simulation is exactly the high-tail, zero-fill corner of our masking family. If the extra bands and the aliasing-like fill do not help, our method reduces to it. The experiment that settles this is a matched comparison on the spacing sweep and on native thick cases, especially those with inter-slice gaps where aliasing is expected. If thick-slice simulation is statistically indistinguishable, we report that cheap physical augmentation, not architecture, explains most anisotropy robustness. That result is still useful, but it would mean the frequency-masking claim fails.
Why not just use Cutout or random slice dropout? SpecAugment's finding is that convolutions interpolate around scattered holes, so contiguous band structure is what forces redundancy (SpecAugment. Dropping slices is spatially contiguous, but it alters every through-plane frequency at once rather than a selected band. Our matched-volume ablation tests whether the frequency-band component adds anything beyond spatial contiguity.
Why not just use a 2.5D or cross-slice-attention architecture? These help: CSAM reports 51.0% Dice against 37.0% for 3D nnU-Net on prostate cancer (CSAM). But they are trained at a fixed spacing distribution and say nothing about robustness when the test spacing shifts. Our augmentation is orthogonal to them, so we test it both as a replacement (on plain nnU-Net) and as an addition (on MNet).
Proposed experiments
The primary benchmarks are LiTS and the Decathlon brain tumor task. LiTS has varied native spacing and MNet's published 1 mm to 5 mm degradation result. The brain tumor data have near-isotropic volumes from which thick-slice test sets can be derived cleanly. KiTS serves as a second abdominal CT check. PROMISE12 serves as a natively anisotropic MRI check, where CSA-Net reports 0.921 Dice (CSA-Net). We report Dice, HD95, and average symmetric surface distance at each test spacing. We also report two summary measures: the area under the Dice-versus-spacing curve, and the Dice drop from 1 mm to 5 mm. Lesion-level Dice is stratified by lesion size, to catch the small-target destruction the audio literature warns about.
The first decisive check is whether full masking beats matched thick-slice simulation on the 5 mm and gapped test sets without hurting native-spacing Dice. The second is whether the aliasing-like fill beats zero-fill specifically on gapped acquisitions, which is the physical prediction. The third is whether spectral sensitivity profiles show concentrated band reliance in vanilla nnU-Net that flattens after training. If the first check fails, the proposal reduces to a careful benchmark of physical augmentation. If the second or third fails, the mechanistic story is wrong even if accuracy improves, and we will say so.
References
- MNet: Rethinking 2D/3D Networks for Anisotropic Medical Image Segmentation
- Automatic Brain Tumor Segmentation Using Cascaded Anisotropic Convolutional Neural Networks
- Automatic Segmentation of Vestibular Schwannoma from T2-Weighted MRI by Deep Spatial Attention with Hardness-Weighted Loss
- CAT-Net: A Cross-Slice Attention Transformer Model for Prostate Zonal Segmentation in MRI
- CSAM: A 2.5D Cross-Slice Attention Module for Anisotropic Volumetric Medical Image Segmentation
- A flexible 2.5D medical image segmentation approach with in-slice and cross-slice attention
- Slice Imputation: Intermediate Slice Interpolation for Anisotropic 3D Medical Image Segmentation
- CARVE: Cross-Slice Anisotropic Reallocation of Visual Evidence for Efficient 3D Medical Volume Understanding
- Spatial-Frequency Dual Domain Attention Network For Medical Image Segmentation
- AMLP: Adaptive Masking Lesion Patches for Self-supervised Medical Image Segmentation
- AnatoMask: Enhancing Medical Image Segmentation with Reconstruction-Guided Self-masking
- SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
- Frame-Level Specaugment for Deep Convolutional Neural Networks in Hybrid ASR Systems
- Combining Simple but Novel Data Augmentation Methods for Improving Conformer ASR