TL;DR
SCoRe is a training-free framework that achieves state-of-the-art results in cross-domain few-shot segmentation by intelligently fusing two "experts": a DINO-based model for localization and a SAM-based model for boundary precision. By using support images to calibrate which model should lead for a specific concept, it achieves a 5.2% mIoU gain over standalone methods across diverse domains like medical imaging and satellite photography.
What Problem Does This Paper Address?
Cross-Domain Few-Shot Segmentation (CD-FSS) is the task of segmenting a new object class in a target domain (e.g., finding a tumor in an ultrasound) using only a handful of labeled examples ("supports").
This is notoriously difficult because models trained on natural images (like cats and cars) often fail to generalize to specialized domains like X-rays or industrial defect detection. While modern Vision Foundation Models (VFMs) like DINO and SAM have incredible zero-shot capabilities, they have complementary flaws:
- DINO-based models (like RSRM) are excellent at locating the correct region based on texture and structure but produce "blobby," imprecise boundaries.
- SAM-based models (like FSS-SAM3) produce razor-sharp contours but often segment the wrong object or include "hallucinated" false positives because they lack a strong semantic understanding of non-natural domains.
The authors aim to fuse these models without any additional training or fine-tuning, which is often too slow or data-intensive for real-time few-shot applications.
Methods
SCoRe (Support-Calibrated Role fusion) operates on the principle that for any given task, one model should be the "leader" and the other the "refiner."
1. Support-Only Route Selection
Before processing a query image, SCoRe performs a "leave-one-out" cross-validation on the $k$ support images. It tests two different processing routes and selects the one that performs best on the known support labels. This "freezes" the strategy for the subsequent query inference.
2. The Two Routes
- Region-Led (R-LED): Used when the DINO-based expert is more reliable at finding the object. RSRM provides the base mask, and SAM3 refines the contours. This refinement is restricted to a "boundary band" (8–32 pixels) to ensure the refiner doesn't accidentally relocate the object to a different part of the image.
- Boundary-Led (B-LED): Used when the SAM-based expert provides a better initial shape. SAM3 provides the base mask, and RSRM acts as a "structural guard." RSRM is used to prune regions it confidently identifies as background, fill holes that it identifies as foreground, and bridge gaps between fragments.
3. Deterministic Correction
The fusion isn't learned; it's governed by deterministic rules based on confidence margins and morphological operations (like dilation and erosion). This makes the system predictable and avoids the need for backpropagation.
Datasets
The authors evaluated SCoRe on a massive benchmark of 10 diverse datasets comprising 3,314 five-shot episodes:
- Medical: Chest X-ray, ISIC 2018 (Skin), Kvasir-SEG (Polyps), BUSI (Breast Ultrasound).
- Industrial/Road: KSDD2 (Surface defects), Pothole.
- Natural/General: Oxford-IIIT Pet, Pascal-5i, FSS-1000.
- Aerial: DeepGlobe (Satellite imagery).
Key Insights and Ideas
- Calibration over Training: The "aha moment" is that you don't need to train a fusion network if you have support images. The support set acts as a tiny, domain-specific validation set that tells you which model's "priors" are currently more useful.
- Semantic vs. Geometric Priors: DINO (self-supervised) learns general-purpose features that respond to texture, making it robust to domain shifts. SAM (supervised on masks) learns what an "object" looks like geometrically. SCoRe treats these as two different sensors that need to be calibrated against each other.
- The "Structural Guard" Concept: In the B-LED route, the authors realized that SAM's biggest weakness is "inventing" objects. By using DINO's confidence scores to prune these inventions, they keep SAM's sharp edges while inheriting DINO's semantic sobriety.
Key Results
- Quantitative: SCoRe achieved an average foreground mIoU of 72.738%, a significant 5.210 percentage point gain over the previous state-of-the-art (RSRM).
- Consistency: It provided the best point estimate on 7 out of 10 datasets.
- Efficiency: On an RTX 3090, the system takes roughly 1.36 seconds per query, making it viable for interactive use.
- Support Scaling: Interestingly, performance peaked at 5 support images; increasing to 8 supports did not provide a statistically clear improvement, suggesting that the "route selection" logic saturates early.
Limitations and Future Work
- Inference Latency: Because SCoRe requires running two heavy foundation models (DINOv3 and SAM3) plus the calibration step, it is computationally more expensive than a single-model approach.
- Calibration Reliability: If the support images are very different from the query image (e.g., different lighting or angles), the "leave-one-out" selection might choose the wrong route.
- Correction Risks: While the rules are conservative, there is no mathematical guarantee that an edit will always improve the mask.
- Future Directions: The authors suggest exploring more sophisticated route selection methods and developing "safer" correction rules that can handle cases where both experts are struggling.
Summary updated