SCoRe: Support-Calibrated Role Fusion for Training-Free Cross-Domain Few-Shot Segmentation

Abhishek Kathpal, Adarsh Jagan Sathyamoorthy, Utsav PatelarXiv 2026 · arXiv

TL;DR

SCoRe is a training-free framework that achieves state-of-the-art results in cross-domain few-shot segmentation by intelligently fusing two "experts": a DINO-based model for localization and a SAM-based model for boundary precision. By using support images to calibrate which model should lead for a specific concept, it achieves a 5.2% mIoU gain over standalone methods across diverse domains like medical imaging and satellite photography.

What Problem Does This Paper Address?

Cross-Domain Few-Shot Segmentation (CD-FSS) is the task of segmenting a new object class in a target domain (e.g., finding a tumor in an ultrasound) using only a handful of labeled examples ("supports").

This is notoriously difficult because models trained on natural images (like cats and cars) often fail to generalize to specialized domains like X-rays or industrial defect detection. While modern Vision Foundation Models (VFMs) like DINO and SAM have incredible zero-shot capabilities, they have complementary flaws:

  • DINO-based models (like RSRM) are excellent at locating the correct region based on texture and structure but produce "blobby," imprecise boundaries.
  • SAM-based models (like FSS-SAM3) produce razor-sharp contours but often segment the wrong object or include "hallucinated" false positives because they lack a strong semantic understanding of non-natural domains.

The authors aim to fuse these models without any additional training or fine-tuning, which is often too slow or data-intensive for real-time few-shot applications.

Methods

SCoRe (Support-Calibrated Role fusion) operates on the principle that for any given task, one model should be the "leader" and the other the "refiner."

1. Support-Only Route Selection

Before processing a query image, SCoRe performs a "leave-one-out" cross-validation on the $k$ support images. It tests two different processing routes and selects the one that performs best on the known support labels. This "freezes" the strategy for the subsequent query inference.

2. The Two Routes

  • Region-Led (R-LED): Used when the DINO-based expert is more reliable at finding the object. RSRM provides the base mask, and SAM3 refines the contours. This refinement is restricted to a "boundary band" (8–32 pixels) to ensure the refiner doesn't accidentally relocate the object to a different part of the image.
  • Boundary-Led (B-LED): Used when the SAM-based expert provides a better initial shape. SAM3 provides the base mask, and RSRM acts as a "structural guard." RSRM is used to prune regions it confidently identifies as background, fill holes that it identifies as foreground, and bridge gaps between fragments.

3. Deterministic Correction

The fusion isn't learned; it's governed by deterministic rules based on confidence margins and morphological operations (like dilation and erosion). This makes the system predictable and avoids the need for backpropagation.

Datasets

The authors evaluated SCoRe on a massive benchmark of 10 diverse datasets comprising 3,314 five-shot episodes:

  • Medical: Chest X-ray, ISIC 2018 (Skin), Kvasir-SEG (Polyps), BUSI (Breast Ultrasound).
  • Industrial/Road: KSDD2 (Surface defects), Pothole.
  • Natural/General: Oxford-IIIT Pet, Pascal-5i, FSS-1000.
  • Aerial: DeepGlobe (Satellite imagery).

Key Insights and Ideas

  • Calibration over Training: The "aha moment" is that you don't need to train a fusion network if you have support images. The support set acts as a tiny, domain-specific validation set that tells you which model's "priors" are currently more useful.
  • Semantic vs. Geometric Priors: DINO (self-supervised) learns general-purpose features that respond to texture, making it robust to domain shifts. SAM (supervised on masks) learns what an "object" looks like geometrically. SCoRe treats these as two different sensors that need to be calibrated against each other.
  • The "Structural Guard" Concept: In the B-LED route, the authors realized that SAM's biggest weakness is "inventing" objects. By using DINO's confidence scores to prune these inventions, they keep SAM's sharp edges while inheriting DINO's semantic sobriety.

Key Results

  • Quantitative: SCoRe achieved an average foreground mIoU of 72.738%, a significant 5.210 percentage point gain over the previous state-of-the-art (RSRM).
  • Consistency: It provided the best point estimate on 7 out of 10 datasets.
  • Efficiency: On an RTX 3090, the system takes roughly 1.36 seconds per query, making it viable for interactive use.
  • Support Scaling: Interestingly, performance peaked at 5 support images; increasing to 8 supports did not provide a statistically clear improvement, suggesting that the "route selection" logic saturates early.

Limitations and Future Work

  • Inference Latency: Because SCoRe requires running two heavy foundation models (DINOv3 and SAM3) plus the calibration step, it is computationally more expensive than a single-model approach.
  • Calibration Reliability: If the support images are very different from the query image (e.g., different lighting or angles), the "leave-one-out" selection might choose the wrong route.
  • Correction Risks: While the rules are conservative, there is no mathematical guarantee that an edit will always improve the mask.
  • Future Directions: The authors suggest exploring more sophisticated route selection methods and developing "safer" correction rules that can handle cases where both experts are struggling.

Summary updated

Abstract

We present SCoRe (Support-Calibrated Role fusion), a framework to utilize two complementary vision foundation model-based (VFM) segmenters for training-free cross-domain few-shot segmentation (CD-FSS). Our approach is based on the observation that current VFMs for CD-FSS possess capabilities where one model locates the correct object, while another model computes the better contour. Our approach uses: 1. a method to identify a VFM segmenter to lead Few-Shot Segmentation (FSS) task for a given concept, 2. a novel deterministic boundary correction method using bands of near- and far-pixels at the lead segmenter's predicted boundary of an object, and 3. a region correction mechanism for the other segmenter to conservatively prune components, fill holes, and bridge short gaps on the lead segmenter's predicted masks to improve accuracy. We evaluate SCoRe using RSRM and FSS-SAM3 across 3,314 matched five-shot episodes from 10 datasets. SCoRe achieves 72.738% foreground mIoU averaged across datasets, a gain of 5.210 percentage points over RSRM (paired 95% CI [4.618, 5.972]), with per-dataset gains up to 8.461 points over the strongest standalone method and the best point estimate on 7 of 10 datasets.
BibTeX
@misc{kathpal2026score,
  author = {Abhishek Kathpal and Adarsh Jagan Sathyamoorthy and Utsav Patel},
  title = {{SCoRe: Support-Calibrated Role Fusion for Training-Free Cross-Domain Few-Shot Segmentation}},
  year = {2026},
  eprint = {2611.20582},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20582},
}
Figure