Whole-Scan, Low-Resolution Segmentation of Low-Contrast Organ Boundaries

When an organ such as the pancreas or esophagus has nearly the same CT intensity as the fat and muscle around it, a segmentation network cannot find its edge by looking for a sharp change in brightness, because no such change exists. A radiologist places the edge by knowing where the organ should be: its shape, what it touches, and how it continues across neighboring slices. Current 3D segmentation models handle this poorly. To fit in GPU memory they work on small full-resolution crops, so they see sharp local texture but lose the patient-level context that decides ambiguous edges. This proposal tests the opposite bet. A network runs over the entire scan at low resolution, so every voxel sees the whole anatomy. It then recovers boundaries finer than its own grid by predicting, for each coarse voxel, a small block of signed distances to the organ surface. These distances are unpacked into full resolution only at the last layer.

The claim has two parts, and the second is what makes it worth testing. The engineering part is that a 3D, anisotropic version of the "sub-pixel shuffle" from image super-resolution can carry boundary geometry. A sub-pixel shuffle means predicting several fine-grid values as channels of one coarse cell and then rearranging them in space. The scientific part is a falsifiable prediction about where segmentation accuracy comes from. For low-contrast boundaries, field of view should matter more than resolution, so a whole-scan low-resolution model should match or beat full-resolution crop models on surface distance. For high-contrast boundaries such as kidney or liver, the reverse should hold. If that crossover appears, the three literatures can be reconciled: heavy processing can move to low resolution exactly where local intensity carries little boundary information, and nowhere else.

Where This Sits

Three bodies of work set up the problem. Low-resolution processing in 2D restoration shows that feature extraction can live entirely in a coarse grid with upscaling deferred to the end. ESPCN does this with a learned periodic-shuffle output layer. SGN shuffles pixels into channels so that downsampling discards nothing, then processes the coarsest level first. UHDformer confines global attention to a compressed bottleneck. All three target 2D natural-image restoration and predict intensities, not geometry. We supply the method they lack in 3D, an anisotropic space-to-depth input and a shuffle output for signed distance, and we test it on a segmentation question they never pose.

On volumetric context, FINE adds global "volume tokens" on a coarse grid, which remedies the nearsightedness of crop training and is our strongest motivation. Its boundary gains come only from context, with no explicit boundary output. nnFormer places global attention at the low-resolution bottleneck but upsamples logits uniformly. OSS-Net is the closest existing system. It encodes the whole volume coarsely, adds coordinate-conditioned local patch codes, decodes occupancy with an MLP, and samples densely only in a heuristic band around a rough draft. It fits 512³ volumes in under 16 GB and reports Dice of 0.8842 on BraTS 2020, against 0.8827 for a 3D residual U-Net.

We differ from OSS-Net in three ways. First, our full-resolution signal is a signed distance decoded by a fixed rearrangement rather than an occupancy query. Second, we deliberately include no local high-resolution patch encoder in the main model, which is what tests whether resolution is needed at all. Third, our evaluation targets low-contrast abdominal and thoracic organs, which OSS-Net does not report.

On boundaries, the methods split by dimension. DB-MSMUNet uses an edge decoder branch and reaches 89.47% Dice on NIH Pancreas, but works on independent 2D slices, and its authors flag 3D consistency as unaddressed. Structure Boundary Preserving Segmentation is likewise 2D-only. PnPNet sharpens confused 3D boundaries through feature pushing and prototype pulling, but runs densely on patches. EMSN refines the esophagus with a three-stage full-resolution V-Net cascade. These papers motivate the problem and also constrain the design. Roth et al. showed that 2D predictions need 3D smoothing to reach 68% Dice, so slice independence is not an option. The DB-MSMUNet and EMSN results warn that aggressive downsampling can erase thin edges. Our design answers that warning with lossless space-to-depth input and sub-voxel distance output, and tests it directly.

Core Mechanism

The input CT volume is first rearranged by anisotropic space-to-depth with factors $(r_x, r_y, r_z)$, typically $(4,4,1)$ for 5 mm slices or $(4,4,2)$ for thin slices. This rearrangement stacks each $r_x r_y r_z$ block of voxels into channels, so no raw intensity is discarded. The first layer is kept wide enough (at least $r_x r_y r_z$ channels) that it is not secretly a lossy strided convolution.

A 3D U-shaped body with nnFormer-style local and bottleneck attention then processes the whole scan at once. For a 512×512×200 CT at factor 4 in-plane, that is a 128×128×200 grid. The output at each coarse voxel $v$ is a vector of $K \cdot r_x r_y r_z$ values, where $K$ is the number of organs. These are rearranged into a full-resolution signed distance map $\hat d_k(x)$, which estimates the distance from fine voxel $x$ to the surface of organ $k$ and is negative inside the organ. The training objective is

$$ \mathcal{L} = \mathcal{L}_{\text{Dice}}\big(\sigma(-\hat d / \tau),\, y\big) + \lambda \sum_{k} \frac{1}{|B_k|}\sum_{x \in B_k} \big| \hat d_k(x) - d_k(x) \big|, $$

Here $y$ is the ground-truth mask, $\sigma$ the sigmoid, $\tau$ a temperature in millimetres, and $d_k$ the true signed distance clipped at a band of 5 mm. $B_k$ is that band around the true surface of organ $k$, so the distance term spends its weight only near boundaries, and $\lambda$ balances the two terms.

Two features carry the main bet. First, every voxel's prediction is conditioned on the full scan, never a crop. Second, sub-voxel boundary position is regressed rather than obtained by interpolating coarse logits. Because the $z$ factor is matched to slice spacing, boundaries are predicted jointly across slices. We also measure inter-slice boundary flicker directly rather than assuming it is small.

Research Program

The work has three phases.

The first phase builds the whole-scan model and two matched controls. The first control uses the identical body but upsamples coarse logits with transposed convolutions. The second takes the identical architecture at full resolution, trained on 128³ crops.

The second phase is the analysis that gives the project its scientific content. Every ground-truth organ surface in the test sets is stratified by a measured boundary contrast,

$$ c_k = \frac{\big|\,\bar I_{\text{in}}(B_k) - \bar I_{\text{out}}(B_k)\,\big|}{\hat s_{\text{noise}}}, $$

$\bar I_{\text{in}}$ and $\bar I_{\text{out}}$ are mean Hounsfield values in 2 mm shells just inside and just outside the surface, and $\hat s_{\text{noise}}$ is the intensity standard deviation in a homogeneous region of the same scan. We then fit how surface error depends on resolution factor and field of view, separately within contrast strata.

The third phase is conditional on a crossover appearing. We add a thin full-resolution correction pathway that runs only where $c_k$, estimated locally from the image and coarse prediction, exceeds a learned threshold. This tests whether spending high-resolution compute according to contrast beats spending it according to prediction uncertainty, which is the heuristic OSS-Net uses.

Baselines

The naive reference is a standard full-resolution 3D U-Net trained on crops with Dice and cross-entropy loss. We use the nnU-Net configuration that FINE reports at 87.0% average Dice and 10.5 mm HD95 on Synapse (FINE). It is the default practitioners would deploy, and it tells us whether any of this machinery is needed.

The strongest context-based competitor is the FINE transformer itself, which reports 87.1% Dice and 9.2 mm HD95 on Synapse, the best numbers in the file for that benchmark. It is the most direct rival because it also brings whole-scan context into a high-resolution model. We will rerun it with released settings on our identical folds. To beat it we require lower HD95 and ASSD on pancreas and gallbladder at no more than its GPU memory, with average Dice within one point.

For the pancreas specifically, the strongest number in the file is DB-MSMUNet's 89.47% Dice on NIH Pancreas, with 87.59% on MSD Pancreas. Those figures come from 2D slice evaluation, so we will retrain it under our 3D volumetric Dice protocol rather than compare against the published values. We also report inter-slice flicker, where we expect it to lose clearly.

For the esophagus, EMSN reports 0.8597 Dice on SegTHOR. We will compare against a reimplemented cascade under equal training budget.

OSS-Net, rerun on the same CT benchmarks, is the closest method-level baseline, since it already combines whole-volume coarse encoding with boundary-focused full-resolution decoding. Beating it on surface distance while dropping its local patch encoder would show that the regressed signed distance, not local high-resolution features, carries the boundary.

Every model gets the same splits, preprocessing, augmentation, an equal GPU-hour training budget, and the same number of hyperparameter trials. All are scored by one evaluation script.

Two ablations isolate the key ingredient. The first swaps the shuffled signed-distance output for upsampled logits, holding everything else fixed. The second trains the whole-scan model on crops of the coarse grid. Together they separate the contribution of sub-voxel geometry from the contribution of field of view.

Why not just…?

Why not just add a boundary loss to nnU-Net? This is the cheapest serious alternative and a real threat on Dice. Our hypothesis says it cannot fix the underlying problem: a crop model has no access to the context that decides low-contrast edges, so sharpening its loss sharpens edges it has already misplaced. The experiment settles this directly. We train nnU-Net with exactly our band-limited distance loss. If it closes the HD95 gap on pancreas, the contribution reduces to the loss, and we will report that.

Why not just use the nnU-Net low-resolution-then-full-resolution cascade, or OSS-Net? Both already pair coarse context with full-resolution refinement. Their refinement stage, however, is local and full-resolution everywhere it runs, which is what EMSN shows is expensive. The factorial analysis tests whether that stage earns its cost on low-contrast surfaces. If cascade and whole-scan model tie on low-contrast organs at very different memory, that tie is itself our main result. If the cascade wins there, the crossover hypothesis is wrong and we will say so.

Why not just run a 2D slice model with an edge branch and smooth in 3D? DB-MSMUNet's 2D numbers are strong. Roth et al., however, show that smoothing recovers coherence only up to a ceiling, and isotropic smoothing rounds off slender structures such as the pancreatic tail. Our flicker metric and the esophagus results will show whether this matters in practice. We define flicker as the slice-to-slice change in predicted contour area and centroid, minus the same quantity in the ground truth.

Decisive Checks

We evaluate on Synapse, splitting low-contrast organs (pancreas, gallbladder, stomach) from high-contrast ones (liver, kidneys, aorta). We add NIH and MSD Pancreas, and SegTHOR for the esophagus. Metrics are Dice, HD95, ASSD, flicker, peak training and inference memory, and latency.

The decisive check is the stratified factorial. We plot ASSD against resolution factor at full and cropped field of view within each contrast stratum $c_k$. The hypothesis survives only if two things hold together. In the lowest-contrast tercile, the whole-scan model at factor 4 matches or beats full-resolution crops on ASSD. In the highest tercile, full-resolution crops win.

The second check is the logit-versus-distance ablation on surfaces within one coarse voxel of the truth. If regressed distances do not reduce sub-voxel error, the 3D shuffle adds nothing over interpolation.

A uniform result would be informative in either direction: resolution winning everywhere would refute the field-of-view claim, and the whole-scan model winning everywhere would challenge the downsampling warnings from the boundary literature.