When a detector looks for a faint pancreatic tumor or a mediastinal mass in a CT volume, it has to decide where to look as well as what box to draw. Current 3D medical detectors decide where to look with a fixed recipe. They tile the volume with predefined boxes, called anchors, whose sizes are tuned once per dataset. They then classify each box from features taken on a rigid cubic grid around its center. When the lesion's edge is barely visible, that rigid grid mixes lesion, healthy parenchyma and vessel into one feature. One common fix is "deformable" convolution, a layer that learns to move its sampling points. In low-contrast images it tends to drift, because nothing in the image tells the points where to go.
We propose to give the sampling points a target. A small head predicts a 3D signed distance field, meaning for every voxel how far it is from the nearest lesion surface, with the sign telling inside from outside. Each candidate box then places its feature-sampling points where that field crosses zero along a set of rays from the box center. The predicted box size only limits how far those rays may extend. In plain terms, the detector reads features at its own best guess of the lesion surface in 3D, instead of on a fixed cube or at points that wander freely. We will test whether this beats self-configured 3D detection with optimized static anchors on low-contrast benchmarks. We will also test whether the gain comes from the geometric steering of the sampling points, and not merely from adding boundary supervision.
Core Claim
In 3D detection of low-contrast lesions, much of the remaining error comes from where box features are sampled, not from the anchor sizes. Tying sampling locations to a predicted volumetric surface should improve sensitivity at matched false-positive rates, especially for small and irregular lesions. It should also make the detector tolerant to strong rotation augmentation, which axis-aligned anchors currently cannot absorb. The sharpest falsifiable prediction concerns an ablation. The same distance-field head used only as an auxiliary loss, with the sampling left rigid or freely deformable, should recover clearly less of the gain than the steered version.
Where This Sits
Three prior systems each supply one piece, and none supplies the coupling.
Region Proposal by Guided Anchoring predicts an anchor shape per location. It then uses deformable convolution, with offsets computed from that shape, so the receptive field matches the anchor. It reports 9.1% higher recall with 90% fewer anchors on MS COCO. It works in 2D on sharp-edged photographs, and its offsets depend only on box width and height, never on where the object's surface actually is. We reuse its idea that the predicted shape should govern sampling, but move it to 3D and add a boundary signal.
nnDetection is the strongest 3D medical detector in the file. It tunes static 3D anchors by differential evolution and uses only rigid 3D kernels. Its MELA extension, Accurate Detection of Mediastinal Lesions with nnDetection, adds ellipsoid pseudo-masks as auxiliary segmentation supervision. It also had to cap rotation augmentation at ±10° because rotated axis-aligned boxes swell with background. That cap is a constraint our design must address directly, and it motivates the rotation experiment.
The boundary side supplies method and motivation.
- Distance-map loss. Boundary loss for highly unbalanced segmentation uses a precomputed signed distance map as a loss on a fixed grid. It improved ISLES Dice from 0.575 to 0.656. We borrow its distance map as a supervision target, but use the prediction to move sampling, not only to penalize outputs.
- Soft box labels. Bounding Maps for Universal Lesion Detection shows that soft, tapered box labels help low-contrast lesion detection in 2D, raising Faster R-CNN sensitivity at 4 false positives per image from 82.43% to 86.28%. Its authors leave 3D open.
- Free deformable convolution plus edge path. DB-MSMUNet combines unconstrained deformable convolution with a separate edge path for low-contrast pancreas. Because it is 2D, it loses through-plane consistency. Its edge branch never informs where its deformable kernels sample, and that separation is exactly what we remove.
- Slice-wise inconsistency. Roth et al. document that slice-wise prediction yields jagged 3D shapes. This is why the distance field and the rays live in full 3D rather than per slice.
Research Program
The base system is the nnDetection Retina U-Net, keeping its self-configuration, anchor search, and post-processing. We add two components at the coarse detection level.
The first is a distance head. It predicts a truncated signed distance field $\hat\phi$ in millimetres. It is supervised with a smooth L1 loss against the true field, plus the boundary loss of Kervadec et al. paired with the existing segmentation loss. The supervision source depends on the dataset:
- Where segmentation masks exist, the target field is computed from them. This covers datasets that nnDetection converts from segmentation challenges: KiTS19, RibFrac, CADA, and MSD Task07 pancreas.
- On box-only data such as MELA, it is computed from ellipsoid pseudo-masks. This lets us measure how much the method degrades when the surface target is itself a guess.
The second component is a steered sampling module, which replaces the rigid features feeding each anchor's classification and regression. For a candidate with center $c$ and predicted half-extents $e$, we cast $K$ rays in fixed directions $u_k$ defined in physical millimetre space. This choice keeps thick-slice anisotropy from distorting the geometry. The surface radius along each ray is a differentiable soft root of the distance field:
$$r_k = \sum_{t \in T_k} \frac{\exp(-|\hat\phi(c + t u_k)|/\tau)}{\sum_{t' \in T_k}\exp(-|\hat\phi(c + t' u_k)|/\tau)}\, t .$$
Here $T_k$ is a set of candidate distances from zero up to the box half-extent along $u_k$, and $\tau$ is a temperature. Features are trilinearly interpolated at the surface points $c + r_k u_k$, plus a few interior points at fractions of $r_k$. They are then pooled into the head.
Because the module samples $K$ points per location rather than predicting $3N$ free offsets per voxel, it should cost less memory than full 3D deformable convolution. We will report this rather than assume it.
Rays rotate with the anatomy only through $\hat\phi$, which is learned from rotated data. Whether this relieves the rotation cap is therefore an empirical question.
We will also build a diagnostic, which we call the boundary-hit rate. It is the fraction of sampling points lying within one in-plane voxel of the true surface. We compute it for rigid sampling, free deformable sampling, and steered sampling, separately for low-contrast and high-contrast lesions. This directly measures the "offset drift" that motivates the work.
Baselines
Naive reference. The do-nothing reference is nnDetection exactly as released, with differential-evolution anchors, rigid kernels, and its default augmentation. It is retrained with its own configuration on each split.
Strongest existing method. This is the same nnDetection with the MELA-style auxiliary segmentation branch. That branch uses ellipsoid pseudo-masks on box-only data and true masks elsewhere. It is the strongest competitor because it already turns boundary information into a training signal inside a 3D anchor detector, which is our closest prior art. Its reported numbers on these data are:
| Benchmark | Metric | Reported result |
|---|---|---|
| MELA 2022 | FROC at IoU 0.3 | 0.9880 cross-validation, 0.9897 on the leaderboard (Accurate Detection of Mediastinal Lesions with nnDetection) |
| LUNA16 (plain nnDetection) | CPM | 0.930 (nnDetection) |
| KiTS19, RibFrac, CADA | detection | No reported result found on Lacuna |
| MSD pancreas tumor | detection | No reported result found on Lacuna |
For the pancreas tumor, the only figure is a segmentation Dice of 0.21 from The Medical Segmentation Decathlon. That is not comparable to detection. We will therefore produce our own nnDetection numbers under identical folds.
Ablations isolating the key ingredient. Four ablations each remove one part of the design:
- Free deformable. 3D deformable convolution in the head with learned free offsets, in the style of Guided Anchoring, with offsets conditioned on the predicted extents.
- Auxiliary only. Our distance head trained identically, but used only as an auxiliary loss, with rigid sampling.
- Fixed rays. Steered sampling with $r_k$ set to the box half-extent rather than the distance-field root, which tests whether the surface location matters.
- Oracle field. Steered sampling driven by the ground-truth distance field. This gives an upper bound on what better surface prediction could buy.
Fair comparison. All variants share the nnDetection folds, patch size, epoch budget, and post-processing search. They also share the same small hyperparameter budget: eight trials per variant, over learning rate and loss weights only.
What counts as beating the baseline. Steered sampling beats the baselines if it improves FROC-averaged sensitivity over the strongest baseline on the pancreas-tumor and KiTS19 detection tasks. The improvement must exceed the across-fold standard deviation of the baseline. It must also exceed the "auxiliary only" ablation by a margin that holds on small lesions.
Why not just…?
Why not just keep optimized static anchors, as nnDetection does? Differential evolution fixes anchor sizes globally, but it does nothing about where features are read inside an anchor. Guided Anchoring showed that reshaping sampling to match content added up to 2.7 mAP on top of strong detectors in 2D. Static anchors also force the ±10° rotation cap reported on MELA. If plain nnDetection matches us on the pancreas and kidney tasks, and our rotation experiment shows no relief, the proposal fails. These experiments are designed to show exactly that.
Why not just add unconstrained 3D deformable convolution? This is the "free deformable" ablation. The file gives qualitative reasons to expect drift in low contrast, such as DB-MSMUNet's need for a separate edge path. It gives no 3D measurement, so this is a real open question rather than a settled one. The boundary-hit-rate diagnostic and the low-contrast versus high-contrast split will show whether free offsets actually drift. If free offsets reach the same hit rate and the same sensitivity, the distance field is unnecessary, and we will report that.
Why not just add 3D boundary supervision, whether bounding maps, a boundary loss, or pseudo-masks, without moving the sampling? This is the most serious threat. Bounding Maps and the MELA pseudo-masks both show that boundary-aware labels alone give real gains. The "auxiliary only" ablation answers it directly, because it shares every parameter and loss with our method except the steering. If it recovers most of the gain, the honest conclusion is that boundary supervision, not boundary-steered sampling, is the useful ingredient. In that case the work becomes a careful 3D extension of soft-label supervision.
Why not just slice to 2D and use a strong 2D deformable model? DB-MSMUNet's own authors list the loss of through-plane consistency as a limitation. Roth et al. document jagged slice-wise reconstructions. We will not rebuild this pipeline. Instead, we will report how often steered rays at oblique, through-plane directions carry the decisive surface points, which tests whether the 3D component matters.
Proposed Experiments
The main experiment runs every baseline and ablation on four benchmarks:
- MSD Task07 pancreas tumor, converted to detection. This is the primary benchmark: low contrast, with large headroom.
- KiTS19 and RibFrac. Both have masks and use nnDetection's conversion.
- MELA 2022. Box-only, with ellipsoid-derived fields.
On MELA the baseline sits near 0.99, so gains there will be small. We treat MELA mainly as a test of whether guessed surfaces still help, or at least do no harm.
Metrics are FROC sensitivity at 1/8 to 8 false positives per scan, reported at IoU 0.1 and 0.3. Results are stratified by lesion diameter and by lesion-to-surroundings contrast, the latter computed as mean Hounsfield-unit difference across the true boundary. We also report GPU memory and inference time.
Three checks decide the outcome:
- Steering versus auxiliary supervision. The gap between steered sampling and "auxiliary only" should be largest in the lowest-contrast tercile, and it should track the gap in boundary-hit rate.
- Rotation tolerance. Retrain every variant with ±45° rotation. The baseline is expected to lose sensitivity, as the MELA report implies. Steered sampling should lose less.
- Headroom from better surfaces. The oracle-field run tells us how much gain is limited by distance-field quality. If even the oracle barely beats rigid sampling, the core claim is wrong.
References
- Region Proposal by Guided Anchoring
- nnDetection: A Self-configuring Method for Medical Object Detection
- Accurate Detection of Mediastinal Lesions with nnDetection
- Boundary loss for highly unbalanced segmentation
- Bounding Maps for Universal Lesion Detection
- DB-MSMUNet: Dual Branch Multi-Scale Mamba UNet for Pancreatic CT Scans Segmentation
- Deep convolutional networks for pancreas segmentation in CT imaging
- The Medical Segmentation Decathlon