CT and MRI scans are three-dimensional, but many segmentation models read them as a stack of 2D images. The stack can be thin (1 mm between slices) or thick (5 mm), depending on the scanner protocol. Models that mix information across slices usually do so by slice number: "the slice above" and "the slice below." That treats a 1 mm step and a 5 mm step as the same thing. The result is that a model trained on one slice spacing degrades on another. The usual fix is to resample every scan onto an even grid, which invents intermediate slices that were never measured.
EEG and MEG researchers faced a closely related problem, and this proposal borrows their lesson. Electrodes sit at irregular physical positions on the scalp. Models improve when channels are organized by true physical distance rather than by channel number, and they collapse when that geometry is scrambled or ignored. We propose to treat each acquired slice the same way: as a sensor sitting at a known physical depth and covering a known physical thickness. Information is then mixed across slices by a learned function of the distance between them in millimetres, not their index gap. Because the depth coordinate is continuous, the same model can be asked for a segmentation at any depth, including depths between the measured slices. We test whether this makes models robust when slice spacing changes between training and deployment.
Core Claim
The claim is that spacing robustness in 2.5D segmentation should come from how cross-slice mixing is parameterized, not from resampling the volume. A 2.5D model is one that processes slices with a 2D network but lets them exchange information. Concretely, for 2D features $f_j$ extracted from acquired slice $j$ with centre depth $z_j$ and thickness $t_j$ (both in mm), the fused feature at any query depth $z$ is
$$h(z) = \sum_j \operatorname{softmax}_j\big(q(z)^\top k_j + \phi_\theta(z - z_j,\, t_j)\big)\, v_j .$$
In this equation, $q$, $k_j$ and $v_j$ are the usual attention query, key and value projections of the slice features. The term $\phi_\theta$ is a small network that turns physical offset and slice thickness into an attention bias. Nothing in the model sees the integer slice index.
Two predictions follow. First, a model trained at one spacing and tested at another should lose less accuracy than index-based cross-slice models. Second, querying $h(z)$ at held-out depths should produce masks that are more accurate than simply interpolating the masks of the neighbouring slices.
We state the modesty of this plainly. The equation is close to a relative-position attention bias measured in millimetres. The new parts are the thickness term, the ability to query at continuous depths, and a spacing-shift evaluation protocol that isolates which ingredient actually confers robustness.
Relation to Prior Work
Several papers motivate the design by showing that geometry matters on the sensor side. Bashivan et al. project 3D electrode positions onto a plane with a distance-preserving map before building image-like inputs, and find that topology-preserving layouts beat arbitrary ones. Dong et al. 2025 cluster sensors by true 3D coordinates for 3D convolutions, and report that scrambling sensor positions severely hurts spike and seizure detection. Both work on a fixed sensor layout. We carry their lesson to a setting where the "layout," meaning the slice positions, changes from scan to scan.
The same literature also names the failure mode we avoid. Interpolating sensors onto a dense grid fabricates values where nothing was measured, which is the direct analogue of isotropic resampling in CT.
Three papers supply the competing methods. MNet learns to fuse fixed 2D and 3D convolution paths. It shows that 3D U-Net collapses as z-spacing on LiTS is degraded from 1 to 5 mm while MNet stays stable. Its depth handling is still discrete convolution, and it cannot be queried between slices. CSAM adds slice-wise attention with an uncertainty block to 2D U-Nets and beats 3D nnU-Net on anisotropic prostate MRI. It treats slice order as an ordinal sequence, so a change in physical spacing is invisible to it. Our module is designed to replace exactly CSAM's slice attention, which makes the comparison clean.
Neural operator Thompson sampling is only a loose motivation. It shows that operators defined on continuous coordinates can be evaluated at arbitrary resolution, but it concerns black-box optimization on partial differential equation benchmarks, not segmentation. We do not use its method.
Research Program
We build the fusion module as a drop-in replacement for the slice-attention stage of a 2D U-Net++, following CSAM's placement at the skip connections. Each skip connection carries features from the encoder to the decoder, and cross-slice mixing happens there. The decoder can be run at acquired depths or at arbitrary query depths.
The bias network $\phi_\theta$ is a two-layer network on $(\Delta z, t_j)$. It is constrained to be monotone non-increasing in $|\Delta z|$ within a thickness band, so that attention decays smoothly with physical distance rather than jumping at slice boundaries.
Training uses spacing randomization. From thin-slice scans we synthesize thick-slice versions by averaging neighbouring slices under a slice-profile weighting. This mimics how scanners reconstruct thick slices, rather than simply dropping slices. This is the medical analogue of sensor dropout in the EEG literature.
For supervision at non-acquired depths, the original thin slices serve as ground truth. We take a thin scan, build a thick version from it, and ask the model to predict masks at the depths of the thin slices that were averaged away. This gives a self-supervised target and the "between-slice" test described below.
The analysis asks three questions. First, does a model trained on 1 mm data segment well on 5 mm data, and vice versa? Second, does the thickness term matter beyond the distance term? Third, does scrambling the mm coordinates (keeping the order, randomizing the gaps) hurt our model while leaving index-based models unaffected? That last check mirrors Dong et al.'s position-scrambling ablation and verifies that the model actually uses physical distance.
Baselines
The naive reference is a slice-wise 2D U-Net++ with no cross-slice mixing. It sets the floor for through-plane coherence. The pancreas results of Roth et al. show this kind of model producing fragmented 3D shapes.
The resampling baseline is nnU-Net in 3D with isotropic resampling. On LiTS tumors, MNet's paper reports 62.0% Dice for nnU-Net and 51.0% for 3D U-Net.
The strongest existing method on LiTS is MNet, at a reported 66.3% tumor Dice. It is the strongest competitor because it was built specifically for spacing variation and already shows stability under the 1 to 5 mm degradation. On prostate MRI the strongest competitor is CSAM, at a reported 51.0% cancer Dice against 37.0% for 3D nnU-Net. That prostate dataset's availability must be confirmed; if it is unavailable we substitute the PROMISE12 data used by MNet, for which no reported result found on Lacuna.
All methods are retrained by us on identical splits. Each gets the same GPU-hour budget and the same number of hyperparameter trials. Each is also given the same spacing-randomized training data, so that augmentation cannot explain any gap.
The key ablation is CSAM-style slice attention with a sinusoidal encoding of physical depth in mm, plus the same augmentation. It differs from our model only in that depth enters as an additive embedding rather than as a distance-and-thickness attention bias. A second ablation removes the thickness input.
Beating the baselines means the following. At matched in-distribution Dice (within about one point of MNet on LiTS), our model should show a significantly smaller Dice drop under train-thin/test-thick and train-thick/test-thin shifts, with confidence intervals over three seeds and scan-level bootstrapping.
Why not just…?
Why not just resample everything to isotropic voxels and use nnU-Net? MNet's controlled LiTS experiment already shows 3D U-Net degrading sharply as spacing widens. Its reported tumor numbers also place resampled 3D models below MNet. We rerun this under spacing shift rather than relying on the reported numbers. If resampled nnU-Net matches us under shift, the proposal fails on its main claim.
Why not just add a millimetre positional encoding to CSAM and train with slice-dropping augmentation? This is the real threat, and it is why it is our central ablation rather than a strawman. The research file cites Wolf et al. and Bharadwaj et al. as evidence against positional encodings. Neither actually tests depth encodings in segmentation, so we do not lean on them. If the encoding ablation matches us on shift robustness and between-slice prediction, the honest conclusion is that physical coordinates plus augmentation are what matter, and the specific kernel form is not. We would report that as the finding.
Why not just stack three adjacent slices as RGB channels? A three-slice window has a fixed index width, so its physical reach changes fivefold between 1 mm and 5 mm scans. It therefore has exactly the flaw we target. It is cheap, so we include it as a baseline under the same shift protocol.
Why not just use full 3D attention? It is a legitimate upper bound on context. However, CSAM reports CAT-Net at roughly 398M parameters against 36.8M for CSAM U-Net++, and full 3D attention still operates on a resampled grid. We include SwinUNETR only if it fits within the shared compute budget, and we otherwise report it as out of budget.
Proposed Experiments
The primary experiment uses LiTS, which has native slice-thickness variation across its 7 centres. We run three settings: train on thin-slice scans and test on synthesized and native thick scans; the reverse; and a mixed setting. The metrics are liver and tumor Dice, lesion-wise recall, and the Dice-versus-spacing slope. Through-plane coherence is measured by the number of spurious connected components per organ, which is the Roth-style fragmentation.
The between-slice test predicts masks at the depths of withheld thin slices. We compare against linear interpolation of neighbouring predicted masks and against MNet run on resampled input.
The secondary experiment repeats the shift protocol on anisotropic prostate MRI, following CSAM's setting.
The decisive checks are three. First, a smaller spacing-shift drop than both MNet and the mm-encoding ablation. Second, a clear loss when the mm gaps are scrambled, while index-based models are unchanged, which shows the model uses geometry. Third, between-slice masks that beat mask interpolation. If the first check fails against the encoding ablation, the kernel is unnecessary.
References
- Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks Pouya Bashivan
- Automated Detection of Epileptic Spikes and Seizures Incorporating a Novel Spatial Clustering Prior
- MNet: Rethinking 2D/3D Networks for Anisotropic Medical Image Segmentation
- CSAM: A 2.5D Cross-Slice Attention Module for Anisotropic Volumetric Medical Image Segmentation
- Thompson Sampling in Function Spaces via Neural Operators
- Deep convolutional networks for pancreas segmentation in CT imaging
- Your other Left! Vision-Language Models Fail to Identify Relative Positions in Medical Images
- Still an Ineffective Method With Supertrials/ERPs - Comments on "Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features"
- The Liver Tumor Segmentation Benchmark (LiTS)