Uncertainty-Aware Extreme Point Tracing for Weakly Supervised Ultrasound Image Segmentation

TL;DR

This paper introduces a weakly supervised framework for ultrasound image segmentation that requires only four extreme points (top, bottom, left, right) instead of tedious pixel-level masks. By combining the "Segment Anything Model 2" (SAM2) with a novel iterative refinement process guided by uncertainty estimation, the authors achieve performance that matches—and sometimes exceeds—fully supervised models while drastically reducing annotation time.

What Problem Does This Paper Address?

In medical imaging, "gold standard" pixel-level annotations are incredibly expensive. Delineating a single lesion in an ultrasound or CT scan can take a clinician several minutes, making large-scale dataset creation a major bottleneck.

Weakly supervised methods (using bounding boxes or points) are faster but often produce "noisy" pseudo-labels with blurry boundaries. Furthermore, most existing methods treat all pixels equally during training, ignoring the fact that boundaries in ultrasound images are often ambiguous due to low contrast and noise. This paper tackles the challenge of turning sparse, 4-point annotations into high-fidelity segmentation masks that the model can actually trust.

Methods

The authors propose a multi-stage framework called Uncertainty-Aware Extreme Point Tracing (UA-EPT):

1. Initial Pseudo-Labeling: They use the four extreme points to create a tight bounding box, which serves as a prompt for SAM2. This leverages a foundation model's prior knowledge to get a decent starting mask.
2. UA-FGEPM (The Core Innovation): To refine these masks, they developed the Uncertainty-Aware Feature-Guided Extreme Point Masking algorithm. It uses Monte Carlo (MC) Dropout—running multiple stochastic forward passes—to estimate pixel-wise uncertainty.
3. Path Tracing: They combine the gradient (from a Sobel operator) and the uncertainty map into a "cost matrix." They then use Dijkstra’s algorithm to trace the optimal path between the four extreme points. This ensures the boundary follows both high-gradient edges and low-uncertainty regions.
4. Dual-Branch Training: The model is trained using two perturbed versions of the image. They introduce an Uncertainty-aware Scale Consistency (USC) loss, which forces the model to be consistent across scales but "down-weights" the loss in high-uncertainty (ambiguous) regions to prevent the model from learning from noise.
5. Box Alignment Loss: A structural constraint that ensures the predicted mask stays within the bounds defined by the initial extreme points.

Datasets

The framework was evaluated on two public ultrasound datasets:

• BUSI (Breast UltraSound Images): Focused on binary lesion segmentation (benign vs. malignant).
• UNS (Ultrasound Nerve Segmentation): A notoriously difficult dataset where even experts struggle to localize nerve structures due to extremely low contrast.

Key Insights and Ideas

• Foundation Models are Starters, Not Finishers: While SAM2 provides excellent initial cues, it isn't perfect for specialized medical domains. The iterative refinement loop (UA-FGEPM) is what allows the model to surpass the "static" labels provided by the foundation model.
• Uncertainty as a Map: Instead of just using uncertainty to measure confidence, the authors use it as a physical "weight" in a path-finding algorithm. This effectively tells the algorithm: "If you aren't sure about the edge here, look for a path where the model is more confident."
• Selective Regularization: By using the USC loss, the authors acknowledge that not all pixels are created equal. Forcing consistency on a pixel the model is confused about is counterproductive; focusing on "stable knowledge" in confident regions leads to better convergence.

Key Results

The results are impressive, particularly given the minimal annotation effort:

• Surpassing Full Supervision: On the BUSI dataset, the proposed method achieved an IoU of 67.70%, actually outperforming the fully supervised baseline (66.71%).
• Closing the Gap: On the difficult UNS dataset, the method reached 67.39% IoU, nearly matching the fully supervised upper bound of 67.54%.
• Ablation Success: The study showed that adding SAM2 improved IoU by ~7.4 percentage points over a simple box baseline, and the iterative uncertainty-guided refinement added another ~1-2 points of precision.

Limitations and Future Work

• Generalization: While tested on ultrasound, the authors note that the framework needs to be validated on other modalities like CT or dermoscopic images.
• Suboptimal Cases: For extremely challenging images where the 4-point method fails, the authors suggest integrating interactive segmentation (allowing a user to add a few extra "corrective" clicks) as a future direction.
• Computational Overhead: Using Monte Carlo Dropout (20 inference runs per update) increases the computational cost during the pseudo-label update phases, though this is only done at fixed intervals (every 100 epochs) rather than every iteration.

@misc{shi2025uncertainty,
  author = {Lei Shi and Gang Li and Junxing Zhang},
  title = {{Uncertainty-Aware Extreme Point Tracing for Weakly Supervised Ultrasound Image Segmentation}},
  year = {2025},
  eprint = {2510.15666},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.15666},
}