Slice Spacing as Time Step: Recurrent Label-Field Reconstruction for Thick-Slice CT

Hospitals store CT scans at very different slice spacings. Research models are usually trained on thin, dense slices, often around a millimetre apart. Many clinical archives instead hold thick-slice reconstructions, with slices several millimetres apart, each averaging tissue over a slab. When a segmentation model trained on thin slices meets a thick-slice scan, it has lost the through-plane detail it relied on. Small, elongated organs suffer most, and the masks it draws jump erratically from slice to slice. We propose to treat this as a sparse-sensing problem, borrowing from fluid dynamics. In that field, a recurrent network reads a short history of a few point sensors and reconstructs a whole flow field, because the physics couples what is measured to what is not. Here, the acquired slices play the role of the sensors. A recurrent network walks along the body axis, with each step sized by the physical distance between slices. It is trained to output the dense, thin-slice organ masks, including at depths where no slice was acquired.

The bet is that making slice spacing an explicit step size of the recurrence turns a change in acquisition protocol into a change of a known input, not a change in image statistics the model has never seen. A model built this way should degrade gracefully as slices get thicker, where fixed-kernel 3D networks degrade sharply. Because it can be queried at any depth, it also produces masks that are continuous along the body axis by construction.

Core Claim

A bidirectional convolutional recurrent network over the slice axis can recover dense 3D organ segmentations from thick, irregularly spaced CT slices better than a 3D U-Net trained with thickness-simulating augmentation. The improvement should be concentrated on small organs and large spacings. The recurrent network's memory updates are gated by the physical gap $\Delta z$ between acquired slices, and it is supervised on the full thin-slice label volume. The new element is not "a recurrent network along $z$". It is the combination of three things: the spacing-conditioned state update, partial-volume-faithful thick-slice simulation during training, and supervision at unobserved depths. Together these let a single model handle any slice spacing seen at test time. The claim is falsifiable on two fronts. If a 3D U-Net given the same thickness simulation matches it across the thickness range, or if truncating the recurrent memory to a few slices costs nothing, the recurrence is not doing the work.

Where This Sits

The mechanism comes from sparse-sensor reconstruction. Kumar et al. used an LSTM over a rolling window of sensor readings in place of dense sensing. SHRED (Williams et al.) and its reduced-order extension (SHRED-ROM) split the problem into a recurrent encoder of sensor history and a shallow decoder back to the full field. Those works report relative errors as low as 6.65% across shallow-water, Kuramoto–Sivashinsky and Navier–Stokes systems. The extension to moving sensors shows the idea survives irregular sampling. We reuse the encoder/shallow-decoder split and the premise that sequence history substitutes for missing samples. We depart from these works in three ways: we sweep a spatial axis rather than time, the step size is irregular and given explicitly, and the decoded field is a label map rather than a physical quantity. SHRED-ROM's own stated weakness also shapes our evaluation. When the memory window is shorter than the dominant scale, the latent state collapses, so we directly test how far along $z$ the memory actually reaches.

Two papers motivate the target problem. Slice Imputation treats depth like video time and warps between neighbouring slices to synthesize missing ones, with a through-plane smoothness loss. It reaches 87.00% Dice on the Decathlon brain tumour task, against 82.91% for a U-Net and 68.69% for linear interpolation. It uses only local frame-to-frame warping with fixed loss weights and is evaluated within a single domain. Our model instead carries a persistent state across the whole scan and never synthesizes intensities, only labels. AMOS shows that scanner shift hits small organs such as the adrenal glands and duodenum far harder than the liver or spleen. This is exactly the pattern expected when through-plane resolution is lost, and it tells us where to look for gains.

Two results constrain the design. M3DA found that heavy standard augmentation in nnU-Net matches or beats most dedicated adaptation methods. It also found that entropy minimization on top of that augmentation closes 62% of the baseline-to-oracle gap on average. Any proposal that skips a strongly augmented 3D baseline is therefore not credible. The same benchmark shows CT-to-MRI transfer collapsing (0.032 Dice baseline, 0.388 best). Cross-modality shift is a different problem, and we deliberately do not claim to solve it. Swin-UMamba brings linear-cost state-space scans into medical U-Nets but scans only within 2D planes, so it leaves the slice axis unmodelled. Deep Poincaré Map shows that dynamical formulations can transfer across sites, with 0.74 zero-shot Dice from SCD to STACOM against 0.66 and 0.51 for earlier methods. It too works slice by slice.

Dynamic task weighting enters in a limited, honest role. Training mixes two kinds of supervision whose gradients plausibly conflict: loss at acquired slices, which is easy, and loss at unacquired depths, which is hard and grows with spacing. Organ losses of very different sizes add a second imbalance. We test whether dividing each loss by its running average (Lakkapragada et al.) helps over a fixed weighting. We also test a conflict-averse update in the style of CAGrad, which reported a mean per-task drop of 0.20% on NYUv2 against 3.97% for PCGrad. DTCS gives theoretical reason to expect domain-induced gradient conflict. Its loss-ratio domain weighting is the template for weighting across spacing regimes, which we treat as pseudo-domains. If static weights do as well, we will report that and drop the component.

Research Program

The model has three parts. A shared 2D encoder embeds each acquired slice $k$, located at physical depth $z_k$, into a feature map $f_k$. A convolutional gated recurrent unit runs along the slice axis in both directions. Its state is decayed in proportion to the gap before each update:

$$ \tilde h_k = \exp(-\Delta z_k/\tau)\odot h_{k-1}, \qquad h_k = \mathrm{GRU}(\tilde h_k, f_k, \Delta z_k), $$

Here $\Delta z_k = z_k - z_{k-1}$ is the physical spacing in millimetres, and $\tau$ is a learned per-channel length scale. To produce a mask at an arbitrary query depth $z$, the model takes the forward state from the nearest slice below and the backward state from the nearest slice above, each decayed by its distance to $z$. A shallow decoder then combines these with the skip features of the two bracketing slices and outputs organ masks. This is SHRED's shallow decoder adapted to label maps.

Training uses thin-slice AMOS CT. For each volume we sample a target spacing between the native value and several millimetres, plus a random slab offset. We build each "acquired" slice by averaging the thin slices within its slab, which reproduces the partial-volume blur of real thick reconstructions rather than simply dropping slices. The loss is the standard Dice-plus-cross-entropy, applied at every native thin-slice depth. Most of these depths are therefore unobserved positions the model must reconstruct. All runs use the nnU-Net intensity and geometric augmentations, so any gain cannot come from augmentation alone.

The analysis program asks three questions. First, does Dice versus test spacing fall off more slowly than for 3D baselines? Second, does the effective memory extend beyond a 3D convolution's through-plane receptive field? We measure this by truncating the recurrence to windows of 1, 3, 9 slices and to the full scan. Third, do the gains concentrate on small organs and on boundary consistency between adjacent slices?

Baselines

The naive reference point is a 2D U-Net applied slice by slice to the thick scan, with masks copied to the nearest thin-slice depth. It represents the volumetric-context-loss failure in its plainest form. It sets the floor and isolates how much any through-plane modelling buys.

The strongest competitor is a full-resolution 3D nnU-Net. Two versions will be run. The standard version resamples every test scan to the training spacing. The augmented version is trained with exactly our partial-volume thickness simulation, so that "it saw thick slices in training" is not our advantage. It is the strongest competitor because AMOS reports that a 3D-UNet reaches 89.04% mDice on the CT test set, ahead of Swin-UNETR at 86.32%, and M3DA shows this family with augmentation is hard to beat under shift. Both versions get the same training volumes, the same epoch budget, and a matched GPU-hour budget for any hyperparameter search. Our model will be sized to within 10% of the 3D network's parameter count.

We also run Slice Imputation-style preprocessing (Slice Imputation): thick scans are interpolated to thin spacing, then segmented with the thin-slice 3D nnU-Net. This tests whether restoring intensities is enough. As an adaptation add-on, we apply entropy minimization at test time on top of both the best baseline and our model, following the M3DA recipe.

The key ablation removes the spacing conditioning, using a fixed decay and no $\Delta z$ input. A second ablation supervises only at acquired slices. A third truncates the memory window. Beating the competition means a statistically reliable gain over the thickness-augmented 3D nnU-Net in mean Dice on small organs at spacings of 3 mm and above, with no loss larger than half a Dice point at native spacing. We have no reported result on Lacuna for any method on train-thin, test-thick AMOS evaluation, so all baseline numbers in that regime will be produced by us.

Why not just…?

Why not just train nnU-Net with thick-slice augmentation? This is the real threat, and it may well suffice. A 3D network that has seen simulated thick inputs could learn the same partial-volume inversion. We do not argue around this; the thickness-augmented nnU-Net is our main comparison under identical simulation. Our reason to expect a difference is structural. A 3D kernel's through-plane reach is a fixed number of voxels, so after resampling a 5 mm scan to 1 mm spacing, most voxels it reads are interpolated. Our state is instead carried across the scan in physical units. If the memory-truncation ablation shows no gain past three slices, and the augmented nnU-Net matches us across spacings, we conclude that thickness augmentation is the answer and report it as such.

Why not just interpolate the missing slices first? Slice Imputation already shows that naive linear interpolation hurts, dropping to 68.69% Dice against 82.91% with no interpolation. Learned imputation helps, but it hallucinates intensities that a downstream segmenter then trusts. It also works from local neighbours only. Running the learned-imputation pipeline as a baseline directly tests whether reconstructing labels, rather than pixels, matters.

Why not just use a 2.5D model that stacks a few neighbouring slices? A stack of three to five slices is a cheap, fixed-window form of through-plane context. It is exactly the window-truncation ablation of our model with the recurrence removed, so the same experiment answers it. If a five-slice window performs as well as full-scan memory, the recurrent framing adds nothing.

Why not just keep fixed loss weights? GradNorm matched an exhaustive grid search in a single run. AdaMT-Net reports large gains over equal weighting on natural scenes. Neither paper speaks to medical label fields, and nnU-Net's fixed sum is strong in practice. Static weighting is therefore our default ablation. Running-average normalization and the conflict-averse update stay in the method only if they improve unobserved-depth and small-organ Dice beyond seed variance.

Proposed Experiments and Decisive Checks

The primary experiment trains on thin-slice AMOS CT and tests on held-out AMOS CT, re-sliced with partial-volume averaging at 1, 2, 3, 5 and 7 mm. We report mean Dice, small-organ Dice (adrenal glands, duodenum, oesophagus), 95th-percentile Hausdorff distance, and an adjacent-slice consistency score: the Dice between masks at consecutive thin-slice depths, compared against the same quantity in the ground truth. The decisive picture is a curve of Dice against spacing. We expect our model's curve to be flatter than the thickness-augmented nnU-Net's, with the gap widest on small organs. At native spacing it should not fall below the reported 89.04% mDice regime by more than noise.

A second experiment checks that the gain is not an artifact of simulation. We stratify the real AMOS CT test scans by native slice spacing and by scanner, and evaluate without re-slicing. The question is whether the advantage persists on genuinely thick scans and survives the scanner shift that AMOS documents. A third experiment checks generality on the anisotropic Decathlon brain tumour task. The reference there is Slice Imputation's reported 87.00% Dice, run under its single-domain protocol, and our model is applied to the same acquired slices. We will not claim a cross-modality result; the M3DA CT-to-MRI task is excluded by design.

The check that most directly decides whether the recurrence matters is memory truncation. If a recurrent model limited to one or three slices is as good as the full-scan version, the SHRED analogy fails for anatomy and the result reduces to "use thickness augmentation". If full-scan memory helps mainly at large spacings and on elongated small structures, the analogy holds: the history of observed slices carries information about the slices that were never acquired. The spacing-conditioning ablation then shows whether making $\Delta z$ an explicit input is what lets one model cover the whole range of thicknesses.