Finding a 3 mm lung nodule in a CT scan, or a 10-meter fishing boat in a 700-megapixel radar image, is a search problem. The targets fill less than a thousandth of the data. Expensive high-resolution models cannot afford to inspect everything, and cheap low-resolution models often cannot see the targets at all. Current systems either scan everything at full cost or run a fixed "scout, then refine" pipeline that throws away any target the scout misses. A more principled option is to treat each model evaluation as an experiment. The system keeps a running probability map of where targets might be, updates it after every look (including looks that find nothing), and chooses the next look to maximize the number of targets found within a fixed compute budget.
We argue that the question worth asking is narrower than "apply long-horizon planning to medical images." A classical result from optimal search theory applies here: when every look has the same resolution, can jump anywhere in the volume, and has a known detection probability, a greedy rule is already essentially optimal. That rule repeatedly inspects the cell with the highest probability-of-target times detection chance per unit cost. Planning ahead earns its keep only when actions differ in kind. A cheap coarse glance has almost no direct chance of confirming a small target, but it changes where the expensive fine looks should go. We propose a specific acquisition rule. It values each coarse glance by simulating its possible outcomes and computing how much they improve the greedy allocation of the remaining fine-resolution budget. We then test this rule against greedy and fixed two-stage policies on FLARE25, NLSTseg, and xView3-SAR.
Core Claim
Under extreme spatial imbalance, the gap between myopic and non-myopic acquisition (one-step versus multi-step planning) is close to zero for single-resolution search. The gap becomes large only for mixed-resolution search, and it should grow as the budget tightens and the targets shrink. The non-myopic part can be made cheap and exact enough by doing lookahead only over coarse actions and rolling out the fine stage with the greedy rule, which is near-optimal for that stage. This gives a tractable acquisition function for million-voxel grids. It avoids the gradient-based scenario trees of Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Trees, which do not fit discrete patch choices. It also avoids static batch plans in the style of BINOCULARS. As we explain below, a static plan structurally assigns no value to the information a coarse glance carries forward.
Research Program
State and belief. A volume is divided into fine cells, for example 32³ voxels in CT or 256² pixels in SAR. Each cell carries a probability $p_i$ of containing a target. A coarse action runs a downsampled detector over a large region, such as a 128³ block, at cost $c_c$. A fine action runs the high-resolution segmenter on one cell at cost $c_f \gg c_c$.
Each action has a sensor model with three parts: a detection probability, a false-alarm score distribution, and a dependence on lesion size. These are calibrated on held-out data. For the fine model and a negative outcome, the update is the standard Bayesian rule:
$$p_i \leftarrow \frac{p_i(1-d_i)}{1-p_i d_i}.$$
Here $d_i$ is the fine model's detection probability in cell $i$. This is the "negative evidence" mechanism that the FOVEA paper uses for vision-language-model crops. It is absent from GF-Screen, which scores sub-volumes independently and simply discards empty ones. Coarse outcomes produce a score map that reweights the $p_i$ of every cell in the glanced region through a calibrated likelihood ratio. The prior comes from anatomical or geographic location statistics on the training set.
Acquisition rule. The utility is the expected number of distinct targets confirmed by a fine look before the budget $B$ runs out. This follows the argument in Amortized Decision-Aware Bayesian Experimental Design that design should target the downstream decision, not uniform uncertainty reduction. It also follows Ostmeier et al., who show that voxel-boundary overlap scores are unstable for sub-milliliter lesions.
For a fine look, the value is the greedy index $p_i d_i / c_f$. For a coarse glance $g$, we sample $K$ fantasized score maps from the sensor model under the current belief. For each sample, we update the belief, compute the expected detections of the greedy fine allocation of the remaining budget $B - c_c$, and then average. The glance value is
$$\mathrm{GV}(g) = \frac{\mathbb{E}_{y\sim g}\big[F(B-c_c \mid y)\big] - F(B)}{c_c},$$
where $F(b \mid y)$ is the expected detections from greedily spending budget $b$ on fine looks after observing outcome $y$. At each step the agent executes whichever action, coarse or fine, has the higher value per unit cost.
The greedy fine allocation is a sort over cells. So the cost of evaluating one glance grows linearly with $K$ times the number of cells, not exponentially with the planning horizon. We will also test a depth-two variant that looks two coarse glances ahead, to measure whether deeper coarse planning helps. This addresses the compounding-error concern raised for long horizons.
Analysis. We will show formally that $\mathrm{GV}$ coincides with the greedy rule when only one resolution exists. We will also characterize when $\mathrm{GV}$ is large: when coarse detection probability is high for regions but low for confirming single targets, which is FOVEA's "information cliff". Finally, we will measure how sensitive the method is to a miscalibrated sensor model, which is the main way planning over simulated outcomes fails.
Baselines
The naive reference point is exhaustive sliding-window inference with the fine model. It is the recall ceiling, and its cost defines a budget fraction of 1.0. It is not a competitor at tight budgets, but every other method is reported as a fraction of its recall. GF-Screen reports that exhaustive evaluation on FLARE25 takes over 100 s per scan.
The strongest existing method is GF-Screen. It uses a ResNet-18 "glance" model trained with group-relative policy optimization to select sub-volumes, followed by a SwinUNETR "focus" segmenter. On FLARE25 it reports +25.6% DSC and +28.2% NSD over the 2024 challenge champion at 28 s per scan. No absolute per-lesion detection rate for GF-Screen was found on Lacuna. We will retrain it with the same coarse and fine backbones as our method. We then sweep its patch-selection threshold so that its compute matches each of our budget fractions. Beating it means higher lesion-level sensitivity at equal compute and equal false positives per scan, with the gain concentrated on lesions under 5 cm³.
On xView3-SAR, the reference is the winning "BloodAxe" model. Paolo et al. report an aggregate score of 0.6177 and detection F1 of 0.7702, against 0.1904 for the Faster-RCNN baseline. We use it as the fine model and ask how much of its score survives at 10–30% of its compute.
Two baselines isolate the key ingredient. The first is the fixed two-phase pipeline in the style of D-S-Net: coarse glance everything, then fine-inspect the top-k. The second is greedy belief-based search over the same mixed action set, which values coarse glances only by their direct detection probability. That second baseline shares our belief model, our negative-evidence updates, and our cost accounting, so any gap is attributable to the lookahead term alone.
Further ablations remove the negative-evidence update, and replace $\mathrm{GV}$ with a static BINOCULARS batch plan. All methods share networks, calibration data, and hyperparameter tuning budget, and compute is counted in network FLOPs.
Why not just…?
Why not just glance everything coarsely and refine the top hits? This is the real threat. When coarse glances are cheap enough to cover the whole volume and coarse detection is reliable, it is close to optimal, and our theory predicts no gain there. Our method should win in two situations. The first is when the budget cannot cover a full coarse sweep plus enough fine looks. The second is when coarse evidence is ambiguous, so repeated or targeted glances pay off. NLSTseg's 500 lesions under 5 cm³ and xView3's 1–2 pixel vessels are such cases. We will report the full curve of budget against gap. If the fixed pipeline matches us across budgets on all three datasets, the claim fails.
Why not just use greedy acquisition with a good belief map? For single-resolution search we expect greedy to be near-optimal, and we will show it. The case against greedy is specific to mixed resolutions. A coarse glance has almost no direct chance of confirming a small target, so greedy undervalues it, which is the "data-barren valley" failure documented by Jiang et al.. The matched-belief greedy baseline settles this directly.
Why not just train an RL scout, as GF-Screen does, or amortize a policy, as Deep Adaptive Design does? An RL scout learns where lesions usually are, but it keeps no belief that responds to what this scan has already shown. Amortized policies such as Deep Adaptive Design are fast. However, Guo et al. find they are brittle when budgets and constraints change, and changing budgets is exactly what our budget sweep varies. Our rule needs no retraining per budget. If time allows, we will distill it into a fast policy and test it on budgets it was not trained on.
Proposed Experiments
We evaluate on FLARE25 (whole-body CT, 0.085% lesion volume, 9 lesion types), NLSTseg (low-dose lung CT, 0.022% tumor pixels), and xView3-SAR (0.005% target pixels, 15-minute per-image limit). Primary metrics are lesion- or object-level sensitivity at fixed false positives per scan, for budget fractions of 0.05, 0.1, 0.2, 0.3, and 0.5 of exhaustive cost. Results are stratified by target size. For xView3 we also report the official aggregate score and detection F1. Segmentation quality is reported with Surface Dice and Volumetric Similarity, which Ostmeier et al. show to be more stable than Dice and well defined on empty scans.
Four checks decide the outcome.
- Single resolution. With only fine actions, $\mathrm{GV}$ and greedy should agree within noise. A large gap here would mean our belief or sensor model is wrong.
- Mixed resolution. The lookahead should beat matched greedy and the fixed two-phase pipeline at budget fractions of 0.3 and below. The gain should shrink toward zero as the budget approaches 1.0 and grow for smaller lesions.
- Negative evidence. Removing the negative-evidence update should hurt most on scans with several lesions, where ruling out regions redirects later looks.
- Miscalibration. We perturb the calibrated detection probabilities by ±30% and check whether the lookahead degrades more steeply than greedy. If so, simulating outcomes from a wrong sensor model costs more than planning gains, which limits where the method should be deployed.
References
- Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Trees
- BINOCULARS for efficient, nonmyopic sequential experimental design
- The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Design
- Glance and Focus Reinforcement for Pan-cancer Screening
- Amortized Decision-Aware Bayesian Experimental Design
- Evaluation of Medical Image Segmentation Models for Uncertain, Small or Empty Reference Annotations
- xView3-SAR: Detecting Dark Fishing Activity Using Synthetic Aperture Radar Imagery
- D-S-Net: an efficient dual-stage strategy for high-precision segmentation of gross tumor volumes in lung cancer CT images
- Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design
- Constrained Bayesian Experimental Design via Online Planning