Keep the Boundary Points That Agree: Consistency-Verified Prompts for One-Shot Segmentation of Faint Objects

Reference-guided segmentation works like this: a user shows a foundation segmenter one annotated example, such as a skin lesion with its mask, and asks it to find the same kind of object in a new image. The usual recipe matches deep features between the example and the new image, then turns the matched locations into "prompt" clicks for the Segment Anything Model (SAM), a general-purpose mask predictor driven by point or box prompts. In medical and satellite images the objects are faint and blend into their surroundings, and this recipe breaks in two ways. Matched points near the faint boundary are unreliable. Sometimes the matcher finds no true matches at all, and then every later step confidently segments the wrong thing.

The strongest current fix, Conditional Point Sparsification (CPS), has two parts. It deletes every matched point near the convex hull of the candidates, and it thins the rest on a grid. This note proposes replacing that geometric heuristic with a test borrowed from robust 3D registration. Each candidate match is kept or discarded according to whether it is mutually consistent with the other matches, and consistency is measured relative to the object's boundary as seen in the annotated example. The same consistency structure also reports when no consistent set of matches exists, so the system can abstain or fall back instead of hallucinating a mask. Put briefly, we stop throwing away boundary points because of where they are and start keeping or rejecting them by whether they agree with each other. The degree of agreement also serves as a signal that the matcher has failed.

Core Claim

Two research threads disagree about boundary points. Matching pipelines discard them because coarse feature maps make edge correspondences jittery. CPS prunes near-hull points explicitly for this reason. Segmentation work on low-contrast targets shows the opposite: boundary-adjacent evidence is exactly what stops masks from leaking. MISNet uses parallel boundary-attention branches with contour supervision. The reverse-attention polyp network erases confident interiors so that capacity goes to the margins.

Our claim is that the tension can be resolved by verifying matches rather than locating them. A boundary match that agrees with its neighbours about how deep inside the object it sits, and in which direction from the object center, is informative and should be kept. A boundary match that disagrees is noise.

The verification borrows the spectral consistency idea of PointDSC. That method builds a graph of pairwise geometric compatibility between candidate 3D correspondences and finds the dominant consistent cluster as the leading eigenvector of the compatibility matrix, computed by power iteration. Rigid distance preservation does not hold between two different lesions, so we replace it with a non-rigid, boundary-anchored compatibility, described below.

The second claim is that the size and strength of the dominant cluster is a usable detector of the "zero-inlier trap." That trap is the failure mode where no backend can recover because the frontend produced no true signal. It is documented for registration consensus in TCF. CPS itself acknowledges that it collapses when DINOv2 matching fails.

Research Program

The pipeline keeps CPS's frozen components: DINOv2 features for matching and a frozen SAM decoder. The method stays training-free and directly comparable.

For each reference point $i$ inside the reference mask, we compute two quantities: its normalized interior depth $d_i \in [0,1]$, the distance to the mask boundary divided by the maximum such distance, and its angle $\theta_i$ about the mask centroid. Each target candidate $i'$ matched to $i$ inherits these as predictions about where it should sit in the unknown target object.

For two candidate matches $(i,i')$ and $(j,j')$, the compatibility is

$$ C_{ij} = s_{ii'}\, s_{jj'} \, \exp\!\Big(-\tfrac{(\Delta d_{ij} - \widehat{\Delta d}_{i'j'})^2}{\sigma_d^2}\Big)\, \exp\!\Big(-\tfrac{(\Delta\theta_{ij} - \Delta\theta_{i'j'})^2}{\sigma_\theta^2}\Big). $$

In this formula, $s$ is the feature similarity of a match, and $\Delta d_{ij}$ and $\Delta\theta_{ij}$ are the differences in reference depth and angle. The term $\widehat{\Delta d}_{i'j'}$ is the corresponding difference in target depth, estimated from each candidate's distance to the hull of the current consensus set. $\Delta\theta_{i'j'}$ is the angular difference about the consensus centroid. Both target quantities are re-estimated over two or three iterations, alternating with the eigenvector solve.

The leading eigenvector $v$ of $C$ scores each match. Matches with high $v$ survive regardless of whether they lie on the hull. The leading eigenvalue $\lambda_1$, normalized by the number of candidates, measures how large and coherent the consistent cluster is. When it falls below a threshold calibrated on the reference image alone, the system flags a matching failure. It does this by re-matching the reference to itself under augmentation, which gives an in-situ estimate of what "success" looks like, much as CPS uses the reference as its density oracle.

Two further stages each address one open problem and are ablated separately.

The first stage is boundary-targeted regeneration, in the spirit of the Progressive Correspondence Regenerator. That method grows new inliers around trusted seed matches. Here, the verified high-depth interior matches serve as seeds. In the band where SAM's multiple output masks disagree, we search locally on the finer DINOv2 patch grid with depth-constrained matching. This is the feedback path that CPS, FCP, and CaPro all lack. In each of them, prompts flow one way into a frozen decoder.

The second stage handles abstention. When a failure is flagged, the system either returns no mask, which is reported in a risk–coverage analysis, or falls back to a reference-free prompt. Both behaviours are measured.

The scope is deliberately narrow. We do not train a boundary network, and we do not use video or 3D geometry. The endoscopic SLAM work (EndoFlow-SLAM, NFL-BA) shows that multi-view photometric constraints help tracking on low-contrast tissue. Extending verification across frames is a natural follow-up, but the single-image few-shot setting has published numbers to beat and is the place to establish whether the mechanism works at all.

Baselines

The naive reference point is dense matched prompting: every mutually-nearest DINOv2 match is fed to SAM without pruning or sparsification. This is the setting CPS shows to collapse on flat cross-domain targets. We also report vanilla SAM, whose weakness on faint structures is documented by CaPro. CaPro reports 15.60% mIoU for vanilla SAM on OCT1600, though that is a different task from ours.

The strongest existing method on this exact question is CPS. It is training-free, uses the same frozen DINOv2 and SAM, and targets the same low-contrast cross-domain benchmarks. CPS reports 1-shot mIoU of 51.3 on ISIC, against 38.6 for the prior training-free baseline, and 70.0 mIoU on Chest X-Ray. No reported result was found on Lacuna for CPS on DeepGlobe or on polyp datasets.

We will rerun CPS from its description with identical backbones, image resolution, and morphological post-processing. Our method gets the same per-dataset tuning budget: the same number of hyperparameter configurations, selected on the same held-out split. Beating CPS means a mean 1-shot mIoU gain on ISIC and Chest X-Ray that exceeds the episode-to-episode standard error over at least 1,000 episodes per dataset, without a loss on DeepGlobe. It should be paired with a larger gain on a boundary-specific metric, Boundary F-score computed on a thin band around the ground-truth contour. A gain confined to interior pixels would not support our claim.

FCP is a secondary baseline. It is the strongest learned prompt-matching method in the file, but it was evaluated only on PASCAL-5i (73.2% 1-shot mIoU) and COCO-20i. We run it on our medical targets to test whether prototype matching transfers to faint domains.

The key ablations isolate the proposal's ingredients. The first keeps CPS's pipeline exactly but swaps hull pruning for our consistency scoring, so the only change is how boundary points are judged. The second uses PointDSC-style spectral scoring with feature similarity only, dropping the depth and angle terms, to test whether the boundary-anchored compatibility matters or any spectral filter would do. Further ablations add regeneration and abstention one at a time.

Why not just…?

Why not just tune or remove CPS's hull-pruning margin? If the only problem were an over-aggressive margin, sweeping it would recover most of the gain. We include this sweep, at the same budget, as a baseline. Our hypothesis is that location is the wrong criterion: some hull points are good and some interior points are bad. The specific prediction is that, at matched numbers of retained points, consistency-selected sets contain more points within a few pixels of the true boundary and yield higher Boundary F-score than any margin setting. If a tuned margin matches us, the consistency machinery is not needed, and we will say so.

Why not just filter with mutual nearest neighbours and a similarity threshold? This is the most serious threat, because cheap match filtering often captures most of the benefit of graph methods. The ablation that keeps similarity and drops the depth and angle terms answers it directly. If that ablation equals the full method, our contribution reduces to "spectral filtering helps," which is weaker. The evidence that it might not suffice is indirect. CPS reports that flat, homogeneous targets yield many high-similarity but redundant or misplaced matches, and similarity alone cannot tell a well-placed boundary match from a misplaced one.

Why not just use SAM's own predicted mask-quality score to pick prompts and detect failure? SAM scores how well a mask fits its prompts, not whether the prompts are on the right object. Under a zero-inlier failure, SAM can return a confident, clean mask of the wrong region. We test this head to head. Both SAM's score and our normalized $\lambda_1$ are used to predict episodes with IoU below 0.3, and we compare the area under the ROC curve. If SAM's score is as good, we drop our failure detector and keep only the selection component.

Why not just use a detector-free dense matcher such as LoFTR? LoFTR avoids keypoint detection and does well on texture-poor natural scenes (65.9% AUC at 3px on HPatches). However, its coarse matching runs at 1/8 resolution, and it is trained for same-scene geometric matching, not for matching across instances of the same category. CPS's results show that adding more dense matches hurts on flat medical targets. We include LoFTR-sourced candidates, passed through both CPS and our verification, as a frontend variant. This tests whether the frontend or the verification is the bottleneck.

Proposed Experiments

The main evaluation follows the CPS protocol: 1-shot and 5-shot episodes on ISIC, Chest X-Ray, and DeepGlobe. Metrics are mIoU and Boundary F-score, with fixed episode seeds shared across all methods. We add a polyp transfer setting with reference and target drawn from Kvasir-SEG and CVC-ClinicDB, and the hard ETIS set as a stress test, because MISNet's results show that small concealed polyps cause large drops for standard models. No reported few-shot result was found on Lacuna for these, so they are exploratory.

Three checks would decide the question.

The first check is the matched-budget comparison against CPS and the margin sweep. Our method must win on Boundary F-score, not only on mIoU.

The second check is a failure-injection study. We deliberately corrupt matching by pairing references and targets from different classes or domains, and by adding blur and contrast reduction. We then measure whether $\lambda_1$ separates successful from failed episodes, with the area under the ROC curve compared against SAM's quality score and mean match similarity. We also report risk–coverage curves for the abstention variant.

The third check is a diagnostic. On held-out episodes, we measure the fraction of retained prompts within a few pixels of the ground-truth boundary that fall inside the true mask, for hull pruning versus consistency selection. This directly tests whether verification keeps the useful boundary points that the heuristic throws away.

If the first two checks fail but the diagnostic succeeds, the mechanism is sound but SAM cannot exploit boundary prompts. That outcome would redirect the work toward learned prompt encoding rather than prompt selection.