ProtoCouncil: Multi-Encoder Anatomical Prototypes for Prostate Cancer Detection in Micro-Ultrasound

Mohammad Mahdi Abootorabi, Nooshin Maghsoodi, Armin Saadat, Obed Dzikunu, Zhuoxin Guo, Hannes Cash, Ferdinand Luger, Adam Kinnaird, Brian Wodlinger, Parvin Mousavi, Purang AbolmaesumiarXiv 2026 · arXiv

TL;DR

ProtoCouncil is a new interpretable framework for prostate cancer detection in micro-ultrasound ($\mu$US) that replaces "black-box" decisions with a "this looks like that" reasoning process. By using a council of three foundation models and grounding predictions in real-world anatomical prototypes, it achieves state-of-the-art accuracy (82.9% AUROC) and provides a built-in safety mechanism to flag uncertain cases for expert review.

What Problem Does This Paper Address?

Prostate cancer (PCa) remains a leading cause of mortality. While MRI-targeted biopsies are the gold standard, they are expensive and resource-intensive. High-resolution micro-ultrasound ($\mu$US) offers a cheaper, real-time alternative, but interpreting these images is notoriously difficult and highly dependent on clinical experience.

Existing deep learning models for $\mu$US often suffer from two major flaws:

  1. Lack of Transparency: They provide a prediction without explaining why, making it hard for clinicians to trust them.
  2. Brittleness: They often fail when encountering data from new hospitals or different biopsy techniques (domain shift).

The authors aim to bridge this gap by creating a model that is both more accurate and inherently interpretable, providing a traceable decision path back to real training examples.

Methods

The core of the paper is the ProtoCouncil architecture, which introduces three key innovations:

  • A "Council" of Foundation Models: Instead of relying on a single encoder, the system uses three distinct foundation models (FMs): MedSAM (trained on medical images), DINOv3 (trained on natural images), and V-JEPA 2.1 (trained on video). The idea is that these models capture complementary features of tissue appearance.
  • Anatomically Aware Prototypes: The model doesn't just look at random patches. It learns "prototypes"—representative features of benign and cancerous tissue—at three specific anatomical levels: the whole gland, the needle track, and six gland sectors.
  • "This Looks Like That" Reasoning: During inference, the model compares the patient's scan to these prototypes. Crucially, these prototypes are "projected" onto real training images. A prediction is literally a weighted sum of similarities to identifiable training cases.
  • Selective Prediction: If the three encoders disagree on a case, the system can "abstain" and flag the core for manual expert review, significantly reducing the rate of false positives and negatives in clinical settings.
Overview of the ProtoCouncil architecture showing the multi-encoder fusion and anatomical prototype heads.

Datasets

The study utilized three distinct $\mu$US cohorts acquired with the ExactVu system:

  • Cohort A (Development): 693 patients (6,588 cores) used for five-fold cross-validation.
  • Cohorts B & C (External): 153 patients (2,018 cores) from the OPTIMUM clinical trial used for zero-shot evaluation. Cohort B used transrectal biopsies, while Cohort C used transperineal, providing a rigorous test of the model's robustness to different clinical procedures.

Key Insights and Ideas

  • Anatomy is a Better Anchor: The authors found that matching prototypes to specific anatomical regions (like the needle track) outperformed standard patch-based prototype methods (like ProtoPNet). This provides a consistent reference frame that clinicians can actually use.
  • Disagreement is a Feature, Not a Bug: The most striking insight is that encoder disagreement is a highly reliable proxy for uncertainty. When all three encoders agree, accuracy is high (79%); when two encoders oppose the fused call, accuracy plummets to 27%. This justifies an "abstain" strategy to improve safety.
  • Foundation Models are Complementary: No single encoder performed best across all cohorts. MedSAM excelled on the development set but struggled on the transperineal cohort (C), where DINOv3 proved more robust. The "council" approach effectively smooths out these individual weaknesses.

Key Results

  • State-of-the-Art Performance: ProtoCouncil achieved an 82.9% AUROC for clinically significant PCa on the development cohort, outperforming the previous best method by 4.3 percentage points.
  • Robustness: It maintained the highest AUROC across all three cohorts, even in zero-shot external testing.
  • Clinical Utility: In qualitative evaluations, an expert confirmed that the model's matched prototypes often shared the same clinical patterns (PRI-MUS scores) as the test cases, validating the interpretability of the "this looks like that" approach.
Qualitative examples showing how test regions are matched to real training prototypes.

Limitations and Future Work

While ProtoCouncil is a significant step forward, the authors note a few areas for improvement:

  • 2D vs. 3D: The current model operates on 2D sagittal frames. Future iterations could integrate information across entire 3D sweeps or multiple cores from the same patient to improve detection.
  • Abstention Rate: The selective prediction strategy is effective but results in the model abstaining on 30-46% of cores. Finding ways to reduce this while maintaining safety is a key challenge for clinical adoption.
  • Vision-Language Integration: The authors suggest that anatomically indexed prototypes could eventually serve as context for vision-language models, allowing for even more sophisticated clinical reporting.

Summary updated

Abstract

Purpose: Micro-ultrasound ($μ$US) is a promising modality for prostate cancer (PCa) detection, but identifying suspicious tissue depends heavily on clinical experience. Models trained on sparse, noisy core-level pathology offer little insight into their decisions and can fail under domain shift. Clinical use needs a traceable decision process and a flag for uncertain predictions. Methods: We introduce ProtoCouncil, in which three foundation encoders (MedSAM, DINOv3 and V-JEPA 2.1) learn case-based prototypes at three anatomical levels: whole gland, needle track and gland sectors. Prototypes are projected onto real training-core regions, and label-free fusion keeps every class logit as a bias plus an exact sum of prototype similarity contributions. Encoder disagreement triggers selective prediction. Results: In five-fold patient-level cross-validation (6,588 cores, 693 patients), ProtoCouncil reaches 82.9% AUROC for clinically significant PCa (csPCa, Grade Group $\geq$3), 4.3 points above the best previously published method. In zero-shot evaluation on two external cohorts, ProtoCouncil achieves higher all-cancer and csPCa AUROC than all previous methods. Nearest same-class exemplars match test-core tumor involvement better than random same-class cases. Externally, csPCa accuracy falls from 79% when encoders agree to 27% when two oppose the fused call. Abstaining on disagreement raises balanced accuracy by 7 and 15 points, while retaining 58% and 54% of the cores. Conclusion: ProtoCouncil grounds core-level predictions in exact similarity-based evidence from real, anatomically indexed training cases, which makes its reasoning interpretable. It achieves the highest csPCa AUROC across three cohorts and identifies unreliable predictions through encoder disagreement.
BibTeX
@misc{abootorabi2026protocouncil,
  author = {Mohammad Mahdi Abootorabi and Nooshin Maghsoodi and Armin Saadat and Obed Dzikunu and Zhuoxin Guo and Hannes Cash and Ferdinand Luger and Adam Kinnaird and Brian Wodlinger and Parvin Mousavi and Purang Abolmaesumi},
  title = {{ProtoCouncil: Multi-Encoder Anatomical Prototypes for Prostate Cancer Detection in Micro-Ultrasound}},
  year = {2026},
  eprint = {2611.09058},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09058},
}
Figure