Organ-Anchored Coordinate Attention for Pancreas Segmentation

Coordinate attention is a cheap way to tell a network where to look. It averages each feature map along image rows and along image columns, then uses the two resulting one-dimensional profiles to reweight every pixel. This works when the object lines up with the image grid. The pancreas does not. It is a thin, curved, C-shaped organ that sits against the duodenum, bowel and fat at nearly the same CT brightness. A row average across it is therefore mostly an average over neighbouring tissue.

We propose to compute the same two profiles in a coordinate system attached to the organ instead of the image. The first coordinate is signed distance from a coarse organ surface, so its profile describes the tissue just inside versus just outside the boundary. The second is position along the organ's centreline, from head to tail, so its profile describes how appearance changes along an organ whose shape varies widely between patients. Every voxel is then reweighted by "how far from the edge am I" and "which part of the organ am I in." These are the two quantities that matter when boundaries are faint and shapes vary.

Core Claim

The claim is narrow and testable. On organs that are both low-contrast and highly variable in shape, coordinate attention helps far more when it pools along organ-anchored coordinates (distance to the boundary, distance along the centreline) than along image axes. The gain should concentrate in a thin band around the boundary and in patients whose organ shape is unusual. The gain should not be explained by simply giving the network a distance map as an extra input, or by rotating the volume so the organ lines up with an axis.

How this sits relative to prior work

The method comes from Coordinate Attention for Efficient Mobile Network Design. That paper factorizes spatial pooling into two directional 1D poolings and reports gains over squeeze-and-excitation on dense prediction (74.0% vs 71.8% on Pascal VOC segmentation). We keep its gating form and change only what the pooling lines are.

Three medical works already use axis-bound versions of this idea, and they motivate the gap. MRA-TUNet applies standard coordinate attention to cardiac MRI, where anatomy is fairly regular. APAUNet projects 3D features onto the sagittal, axial and coronal planes to help with tiny targets. MISNet runs axial reverse and boundary attention along fixed rows and columns of polyp images. All three tie the pooling to the scanner grid. Our proposal ties it to the anatomy.

The nearest attempt at anatomy-attached coordinates is Elastic Boundary Projection. It casts radial rays from interior pivot points and deforms a shell toward the boundary. Its authors report that it breaks down on the pancreas because rays from a pivot assume a star-shaped organ. This result constrains our design. We do not use rays from a point, which assume a roughly convex shape. A centreline plus a distance-to-surface coordinate handles elongated, bent shapes.

Two pancreas-specific networks handle shape variability with adaptive kernels rather than attention. DB-MSMUNet combines deformable convolutions with Mamba state-space blocks on 2D slices and adds an edge decoder. DMC-Net uses dynamically weighted multi-scale kernels whose channel attention pools away all position. Both are strong competitors on the same data. Neither encodes position relative to the organ. The research file notes a known weakness of deformable sampling: its offsets drift in weak-gradient regions. Our coordinates come from a geometric transform of a coarse mask, so they do not depend on local image gradients.

Boundary work motivates the distance coordinate. HF-UNet shows that supervising the prostate boundary with a soft Gaussian band reduces surface error from 3.892 mm to 1.366 mm, compared with treating the boundary as a hard edge. Structure Boundary Preserving Segmentation shows that structured boundary signals beat uniform pixel edge losses, though it relies on an unstable adversarial evaluator. Our distance coordinate gives a deterministic, soft notion of "near the boundary" inside the attention itself, with no discriminator.

Mechanism

A first-pass network produces a coarse probability map of the organ. From its thresholded mask we compute two quantities for every voxel $x$, with gradients stopped through both. The first is the signed distance $d(x)$ to the mask surface. The second is a normalized centreline position $s(x) \in [0,1]$, obtained by skeletonizing the mask, ordering the longest skeleton path from head to tail, and assigning each voxel the position of its nearest skeleton point.

Each coordinate is softly assigned to $K$ bins with weights $w^d_k(x)$ and $w^s_m(x)$ (triangular kernels, summing to one per voxel). For a decoder feature map $X$, the organ-anchored profiles are

$$z^d_c(k) = \frac{\sum_x w^d_k(x)\, X_c(x)}{\sum_x w^d_k(x)}, \qquad z^s_c(m) = \frac{\sum_x w^s_m(x)\, X_c(x)}{\sum_x w^s_m(x)}.$$

These replace the row and column averages of standard coordinate attention. They then pass through the original shared 1×1 transform and sigmoid gates, producing gate vectors $g^d_c(k)$ and $g^s_c(m)$. These are read back at each voxel's own bins:

$$Y_c(x) = X_c(x)\, \Big(\sum_k w^d_k(x)\, g^d_c(k)\Big)\Big(\sum_m w^s_m(x)\, g^s_c(m)\Big).$$

This is exactly the original module whenever the bins are image rows and columns. That property makes the comparison clean.

Two safeguards address the failure mode of reverse attention noted in the file, where a wrong coarse mask poisons the refinement. First, voxels far outside the coarse mask fall into a single "far background" bin. Second, a learned per-stage scalar mixes the organ-anchored gate with an ordinary Cartesian coordinate-attention gate, so the network can fall back when the coarse frame is unreliable. The frame is anchored to the organ, which coarse networks find reliably (organ Dice near 88–89% on NIH Pancreas-CT in the cited papers), not to the tumor, which they often miss.

Research Program

We insert the module into the decoder stages of a 3D nnU-Net. Signed distance and centreline are native 3D quantities, so this directly addresses the cross-slice inconsistency that the 2D DB-MSMUNet leaves open. The coarse mask comes from the same network's first pass, trained jointly with a cascade-style schedule. All other nnU-Net preprocessing, augmentation and post-processing stay fixed.

The analysis has three parts. The first is a boundary-band evaluation: Dice and surface distance restricted to voxels within a few millimetres of the ground-truth surface, further split by local contrast. Local contrast is measured as the Hounsfield-unit difference across the true boundary, so the pancreas–duodenum interface can be examined separately. The second is a morphology stratification: test cases are ranked by ground-truth elongation, centreline curvature, and distance from the training-set mean shape, and the gains are reported by tercile. The third is a robustness sweep: the coarse mask is deliberately eroded, dilated, shifted and partly deleted, to measure how fast the organ-anchored module degrades relative to its Cartesian fallback.

Baselines

The naive reference is a plain 3D nnU-Net trained with its default configuration, with no attention module. The Medical Segmentation Decathlon shows that this kind of self-configuring pipeline is hard to beat with architectural tweaks. Every other variant is built on top of this exact run, with the same folds, epochs and augmentation.

The direct ablation is the same network with standard Cartesian coordinate attention (Hou et al.), extended to three axes, at the same insertion points. A second ablation keeps the organ-anchored module but uses only the distance coordinate or only the centreline coordinate. This tells us which coordinate carries any gain.

The strongest external competitors depend on the target. For NIH Pancreas-CT, the comparison point is DB-MSMUNet's reported 89.47% DSC (DB-MSMUNet). For MSD Task 07 pancreatic tumors, APAUNet reports 69.13 Dice (APAUNet). DMC-Net reports 60.71% tumor DSC and quotes nnU-Net at about 54% (DMC-Net). The MSD overview lists 0.21 DSC for pancreatic tumors (Antonelli et al.), which is clearly not the same protocol. The papers' splits are not guaranteed to match ours, so we will rerun APAUNet and DMC-Net from released code where possible, using our folds and the same GPU-hour budget and tuning trials as our method. Quoted numbers are context, not the comparison.

Beating the baselines means two things. First, a statistically significant gain over the Cartesian-attention ablation in boundary-band Dice and 95th-percentile Hausdorff distance on pancreas. Second, parity or better against rerun APAUNet on tumor Dice. A gain in whole-organ Dice alone, without the boundary-band and shape-tercile pattern, would not count as support for the claim.

Why not just…?

Why not just feed the signed-distance map as an extra input channel to a second-stage network? This is the most serious threat. Any gain could come from the network simply knowing the coarse boundary, not from pooling along it. We include exactly this cascade, with the same coarse mask and the same backbone with Cartesian attention, as a baseline. If it matches our method within noise, the pooling idea is unsupported and we will report that.

Why not just rotate each crop so the pancreas's main axis lines up with the grid, then use ordinary coordinate attention? Aligning by principal axis is cheap. It would fix a straight but tilted organ. It cannot straighten a C-shaped or bent organ, and it gives no distance-to-boundary coordinate. We run it as a baseline. We predict it closes part of the gap on the least-curved tercile and little on the most-curved, and the morphology stratification tests this directly.

Why not just add a boundary loss to nnU-Net? HF-UNet and the COVID-19 network with edge loss (Yu et al.) both suggest that a loss alone gives smaller gains than a loss paired with feature-level guidance. In the latter, edge loss alone gave about 2% DSC, with another 4–5% when combined with attention. That evidence comes from other organs, so we test it here. We train nnU-Net with a soft Gaussian boundary-band loss, with and without our module. If the loss alone captures most of the boundary-band improvement, the module's value is marginal.

Why not just use deformable convolutions, as DB-MSMUNet does? Deformable offsets adapt to shape, but they are known to drift where gradients are weak, which is exactly the low-contrast regime. They also add memory cost. Rerunning DB-MSMUNet-style deformable blocks in our backbone, and comparing boundary-band errors at the lowest-contrast interfaces, settles whether gradient-free organ coordinates are more stable.

Proposed Experiments

The primary benchmark is NIH Pancreas-CT under four-fold cross-validation. Metrics are DSC, 95th-percentile Hausdorff distance, average surface distance, and boundary-band Dice, all stratified by local contrast and shape tercile. MSD Task 07 tests whether an organ-anchored frame helps locate tumors inside the pancreas, reported with tumor Dice and small-lesion detection rate. We treat it as secondary, because the frame is built for the organ.

ACDC serves as a negative control. Cardiac shapes are regular and roughly aligned with the axes, and MRA-TUNet already reports 0.911/0.890/0.961 DSC for right ventricle, myocardium and left ventricle there. We expect little or no gain over Cartesian coordinate attention on ACDC. A large gain there would suggest the effect is generic extra capacity rather than geometry.

Two decisive checks determine whether the idea stands. First, does the organ-anchored module beat both the distance-map-as-input cascade and the axis-aligned rotation baseline, specifically in the boundary band and in the most-curved shape tercile? Second, under the coarse-mask corruption sweep, does performance degrade gracefully to the Cartesian fallback rather than below the plain nnU-Net? If both hold, the paper's claim is that coordinate attention's value depends on choosing coordinates that match the anatomy. If the first fails, we will report that knowing the boundary matters and pooling along it does not.