When a lung tumor presses against the mediastinum, or a lesion has the same density as the tissue around it, the edge of the target cannot be seen from image intensity alone. In medicine, the usual fix is for an expert to trace every edge by hand, and small patient cohorts make that cost impossible to spread out. Cheap supervision in the form of bounding boxes (four numbers per object) has become the standard shortcut. Current box-supervised methods, however, get their boundary information from the image itself: pairwise color similarity, region-intensity homogeneity, or a foundation segmentation model's guess inside the box. Those are the very cues that fail at low contrast. This note proposes getting the boundary from geometry instead. In a 3D scan, a box drawn on a slice touches the object's surface on all four of its sides. Drawing cheap boxes on slices in three orthogonal viewing planes (axial, coronal, sagittal) therefore samples the 3D surface at many points, including exactly where one plane's view is ambiguous. The sides of the boxes also carve away large regions that a single-plane box leaves undecided.
The core claim is concrete. For a fixed annotation budget, boxes spread across three orthogonal planes supervise a volumetric segmenter better than the same number of boxes in one plane. They are also competitive with spending that budget on a few fully traced slices. The advantage should be concentrated in the lowest-contrast cases, because the constraint from box geometry does not depend on intensity differences. If this holds, low-contrast boundary delineation becomes partly an annotation-design problem, which can be solved with small cohorts and no dense masks.
What prior work leaves open
Three lines of work supply the method and show where it stops. Weakly Supervised Instance Segmentation using the Bounding Box Tightness Prior turns a box into a differentiable loss. Every row and column crossing the box must contain at least one object pixel. It pairs this with a pairwise term that assumes neighboring pixels of similar color share a label. We reuse the tightness term and deliberately drop the pairwise term, which that paper's own formulation shows cannot work when lesion and background share intensity. Prompt learning with bounding box constraints for medical image segmentation adapts a SAM-style model from boxes alone, using pseudo-labels from box-prompted SAM, a size constraint, an "emptiness" constraint forcing pixels outside the box to background, and consistency regularization. It works with as few as 20 training images (92.25% Dice on HC18 ultrasound). It is strictly 2D and single-plane, and it has no mechanism for boundaries the underlying model cannot already see. Learning Few-shot Segmentation from Bounding Box Annotations cleans box-contaminated prototypes with a heuristic trimap, but only on natural images with clear silhouettes. None of the three uses boxes from more than one viewing plane of the same object.
Two papers motivate the problem and constrain the design. Can Foundation Models Really Segment Tumors? A Benchmarking Odyssey in Lung CT Imaging finds that box-prompted MedSAM 2 is the best lung-tumor segmenter, but its fine-tuning uses dense masks, and it names box-only adaptation as open. Benchmarking Transfer Learning for Dense Breast Tissue Segmentation on Small Mammogram Datasets reports that MedSAM2 and vision transformers lose to convolutional networks on a 596-image dataset with diffuse boundaries, and that parameter-efficient tuning fails there. We take this as a warning that the foundation model cannot be trusted to find faint edges. It is also why we test a convolutional backbone alongside MedSAM 2, and why diffuse 2D targets like breast density fall outside our scope: the proposal needs a compact 3D object.
Intensity-based boundary priors are ruled out as the primary signal by evidence already in the file. The Chan–Vese active-contour loss in Learning Active Contour Models for Medical Image Segmentation assumes roughly uniform intensity inside and outside the target and was trained with dense masks. Weakly Supervised Lesion Co-Segmentation on Ct Scans found that GrabCut seeds fail at faint boundaries and that dense conditional random field (CRF) smoothing destroyed recall. A Left Ventricular Segmentation Method on 3D Echocardiography Using Deep Learning and Snake shows that geometric priors can rescue ultrasound segmentation from 45 volumes, but its shape model needed full 3D masks.
Core Claim
Let $p_i \in [0,1]$ be the predicted foreground probability at voxel $i$. Consider a box $b$ drawn on a slice in view $v \in \{\text{axial}, \text{coronal}, \text{sagittal}\}$. Let $\bar{B}_b$ be the voxels of that slice outside the box, and let $\mathcal{L}_b$ be the set of in-slice lines that cross the box from side to side. The geometric loss is
$$ \mathcal{L}_{\text{geo}} = \sum_{v}\sum_{b \in v}\Big[\tfrac{1}{|\bar{B}_b|}\sum_{i \in \bar{B}_b} p_i \;-\; \tfrac{1}{|\mathcal{L}_b|}\sum_{\ell \in \mathcal{L}_b} \log \max_{i \in \ell} p_i\Big], $$
The first term is emptiness and the second is tightness in multiple-instance form, where each crossing line must contain at least one foreground voxel. It is added to a down-weighted pseudo-label term from the box-prompted foundation model. With boxes in one plane, the constraints never involve lines running between slices, and they leave the corners of each slice box undecided. Orthogonal boxes add constraints along the slice axis, and their emptiness terms carve the 3D region inside all boxes down to something close to the object's silhouette intersection.
The testable prediction has two parts. The gain over single-plane boxes should grow as boundary contrast falls. And for compact tumors, the carved region plus tightness should leave only a thin undecided band, which a network trained on a small cohort can resolve by pooling shape regularities across cases.
Research Program
The method has three parts. The first is a box-simulation and annotation protocol: from existing 3D masks we generate tight boxes on every $k$-th slice in each chosen view. We add jitter and padding to mimic human error, since the tightness paper shows loose boxes hurt. To soften this sensitivity we relax the max-over-line term with a margin, drawing on the idea of continuous box confidence from Bounding Maps for Universal Lesion Detection. The second is a training recipe with two backbones: MedSAM 2 adapted through a prompt module as in the prompt-learning paper, and a 3D convolutional network trained from scratch, both under $\mathcal{L}_{\text{geo}}$. At test time, both receive only a single 3D box, whose axial projections serve as prompts. This keeps the test-time manual burden identical across methods. The third is a measured annotation cost. Two annotators time box drawing and full contouring on ten held-out volumes, so that budget matching rests on measured seconds rather than on the 15–80× ratio quoted in the tightness paper.
The main analysis sorts cases by a contrast score, the mean intensity difference across the ground-truth boundary divided by local noise. It then reports every method's Dice and 95th-percentile surface distance per contrast tertile.
Baselines
The naive reference point is box-prompted MedSAM 2 used zero-shot. It shows how much a test-time box alone gives. Zero-shot box-prompted SAM 2 is strong on polyps (mDice 0.93 on CVC-ClinicDB in Polyp SAM 2), so it cannot be dismissed. It is run with the same single 3D test box as our method.
The strongest weakly supervised competitor is the box-constraint prompt-learning method of Prompt learning with bounding box constraints, trained on axial boxes only. It is the closest published recipe designed for 20-sample regimes. To keep the comparison fair, it receives the same number of boxes as our method (all placed axially), the same backbone, the same training volumes, and the same hyperparameter search budget. No reported result was found on Lung1 or MSD Task06 for this method, so we reproduce it. Its reported 84.90% multi-dataset average and 92.25% on HC18 serve only to validate our reimplementation on HC18.
The dense-supervision upper reference is fine-tuned box-prompted MedSAM 2 from Can Foundation Models Really Segment Tumors?, which reports Dice 0.9091 on Lung1 and 0.8770 on MSD Task06 using full-cohort dense masks. We rerun it on our reduced cohorts of 5, 10, and 20 volumes so that the gap is measured at equal sample size.
Three ablations isolate the key ingredient. Axial-only boxes at equal count test whether orthogonality, not box count, drives the gain. Orthogonal boxes with emptiness only, or with tightness only, separate carving from surface sampling. Adding a Chan–Vese region term tests whether an intensity prior helps or hurts at low contrast. Beating the prompt-learning baseline means a Dice gain whose paired bootstrap interval over test volumes excludes zero, together with a larger gain in the lowest-contrast tertile than in the highest. Anything less means orthogonality is not the mechanism.
Why not just…?
Why not just carve the hull from the orthogonal boxes and train nnU-Net on it? This is a real threat, because for convex tumors the carved region alone might already be a good mask. We include it directly: the hull, optionally shrunk to satisfy tightness, becomes a pseudo-label for a 3D convolutional network. The GrabCut experience in Weakly Supervised Lesion Co-Segmentation suggests that fixed pseudo-masks lock in their errors. The hull also overestimates concave or lobulated tumors. If hull-plus-nnU-Net matches our trained model, we will report that the contribution is the annotation protocol rather than the loss. That is still a useful finding.
Why not spend the same annotation time contouring a few slices fully and propagate them? Also a real threat. SAM 2's video-style propagation from one prompted frame gave large gains on PolypGen in Polyp SAM 2, and scribble-level sparse labels came within 2.9% of full supervision on cardiac MRI in Learning to Segment Medical Images with Scribble-Supervision Alone. We therefore run a matched arm: 2–3 fully traced axial slices per volume at the measured equal time cost, propagated by MedSAM 2 and used to train the same networks. Our hypothesis is that traced slices fix the boundary precisely where they are drawn but give nothing between slices. The budget-matched comparison on low-contrast tertiles decides this.
Why not keep single-plane boxes and add a boundary loss or CRF cleanup? The evidence is against this at low contrast. The CRF lowered recall in the co-segmentation study, and the pairwise and Chan–Vese assumptions of the tightness-prior paper and the active-contour paper both depend on intensity differences that are missing there. The Chan–Vese ablation and a DenseCRF post-processing arm on the axial-only baseline show this directly.
Proposed Experiments
The main benchmarks are MSD Task06 and NSCLC-Radiomics Lung1, trained on 5, 10, and 20 volumes with the rest held out. CETUS 3D echocardiography is a second, speckled modality where the edges of the heart muscle drop out. HC18 is used only to check the baseline reimplementation. Metrics are Dice, 95th-percentile surface distance, and, on CETUS, mean surface distance, where the snake pipeline's 2.20 mm (end-diastole) gives a reference point for a dense-mask method.
Two checks would settle the idea. The first is the contrast-tertile interaction. Orthogonal boxes must beat equal-count axial boxes by more in the lowest tertile than in the highest, and the gap must not shrink as the cohort grows from 5 to 20. The second is the budget-matched comparison against traced slices plus propagation, done in measured annotator time. If orthogonal boxes win on low-contrast cases at equal cost and against the carved-hull baseline, the claim holds. If the traced-slice arm wins, the finding is that boxes are the wrong currency for faint boundaries, and we will report it that way. A robustness sweep over box padding of 0–10% of object size reports how much the relaxed tightness term preserves.
References
- Weakly Supervised Instance Segmentation using the Bounding Box Tightness Prior
- Prompt Learning With Bounding Box Constraints for Medical Image Segmentation
- Learning Few-shot Segmentation from Bounding Box Annotations
- Can Foundation Models Really Segment Tumors? A Benchmarking Odyssey in Lung CT Imaging
- Benchmarking Transfer Learning for Dense Breast Tissue Segmentation on Small Mammogram Datasets
- Learning Active Contour Models for Medical Image Segmentation
- Weakly Supervised Lesion Co-Segmentation on Ct Scans
- A Left Ventricular Segmentation Method on 3D Echocardiography Using Deep Learning and Snake
- Bounding Maps for Universal Lesion Detection
- Polyp SAM 2: Advancing Zero shot Polyp Segmentation in Colorectal Cancer Detection
- Learning to Segment Medical Images with Scribble-Supervision Alone