Uni-Loc: Weakly Supervised 3D CT Pathology Localization via Spatially Grounded Vision-Language Classification

Hao-Chih Lee, Zahi A. Fayad, Xueyan MeiarXiv 2026 · arXiv

TL;DR

Uni-Loc is a novel vision-language framework that achieves high-resolution 3D pathology localization in CT scans using only scan-level labels. By unifying classification and localization through shared spatial similarity scores, it provides "grounded" interpretability that outperforms traditional post-hoc explainability methods without requiring expensive voxel-level annotations.

What Problem Does This Paper Address?

While deep learning models excel at detecting pathologies in 3D Computed Tomography (CT), they often function as "black boxes." For a radiologist to trust an automated diagnosis, they need to see where the model is looking.

However, creating the "ground truth" maps needed to train models for localization is incredibly labor-intensive, requiring experts to manually annotate hundreds of slices per 3D volume. Existing "weakly supervised" methods—which try to learn localization from simple "sick/healthy" labels—often struggle with the complexity of 3D anatomy. Furthermore, common post-hoc explainability tools (like saliency maps) have been criticized for being unreliable in medical contexts. Uni-Loc addresses this by building spatial grounding directly into the model's architecture.

Methods

Uni-Loc builds upon the Uniferum architecture, a BERT-style multimodal model. The authors introduce three key innovations to bridge the gap between global classification and dense 3D localization:

  1. Query-Conditioned Similarity Pooling: Instead of using a standard classification head, Uni-Loc treats the transformed text-image [CLS] token as a "pathology query." It calculates the cosine similarity between this query and every spatial feature in the image. These similarity scores are then pooled using an attention mechanism to create the final classification logit. This ensures that a high classification score must be backed by strong local spatial evidence.
  2. Token-to-Voxel Upsampling: Transformer models typically reduce spatial resolution significantly (e.g., 32-fold). To recover fine-grained details, Uni-Loc uses a U-Net-style progressive upsampler with 3D voxel shuffling and lateral connections. This allows the model to produce localization maps at the original voxel resolution.
  3. Gradient-Guided Thresholding (GradT): To turn a probability map into a binary "abnormality region," the authors developed a parameter-free rule based on the gradient of the classification logit. This adaptive thresholding identifies voxels that contribute most positively to the model's final decision.

Datasets

The model was trained on CT-RATE, a large-scale dataset of CT scans and radiology reports. For evaluation, the authors used five localization benchmarks comprising 355 expert-annotated scans across four public datasets:

  • GGO: Ground-glass opacity (10 scans)
  • PE: Pleural effusion (78 scans)
  • LAD: Mediastinal lymphadenopathy (120 scans)
  • AS: Atelectasis (35 scans)
  • IO: Infectious opacity (112 scans)

Classification performance was also benchmarked on the external RAD-ChestCT dataset.

Key Insights and Ideas

  • Grounding by Design: By deriving the classification logit directly from spatial similarities, the model is forced to align its "reasoning" with anatomical locations. This is a shift from "post-hoc" explanation to "intrinsic" interpretability.
  • The Failure of Max-Pooling: The authors found that standard max-pooling (often used in weak supervision) produces "degenerate" maps that cluster at image borders. Attention-weighted pooling is essential for meaningful 3D localization.
  • Adaptive vs. Fixed Thresholds: The proposed GradT rule outperformed fixed thresholds (like $p=0.5$) and even rivaled "oracle" thresholds that were manually tuned using ground-truth data. This suggests that the model's internal gradients are a robust signal for delineating pathology boundaries.

Key Results

Uni-Loc demonstrated superior performance in both identifying what is wrong and where it is:

  • Localization Accuracy: It achieved a macro-averaged voxel-level AUROC of 96.1% and AUPR of 34.5%, significantly outperforming traditional methods like Layer-wise Relevance Propagation (LXT) and gradient saliency.
  • Classification Boost: Interestingly, adding the localization constraints didn't hurt classification; it improved it. Uni-Loc reached an AUROC of 88.39% on CT-RATE, outperforming the baseline Uni-CLS model.
  • Qualitative Success: As seen in Figure 1, the model's predicted probability maps (red) show strong spatial correspondence with expert annotations (green) across diverse pathologies.
Visual comparison of Uni-Loc predictions (red) vs. expert annotations (green) for various pathologies.

Limitations and Future Work

While Uni-Loc is a major step forward, the authors note some remaining challenges:

  • Pathology Specificity: Localization remains difficult for certain abnormalities like Atelectasis (AS), which had a much lower AUPR (0.098) compared to Ground-glass opacity (0.449).
  • Prompt Sensitivity: The model currently relies on fixed, manually defined text prompts (e.g., "Diagnosis if there is [pathology] in the lungs"). Future work could explore more flexible or learned prompting strategies.
  • Computational Cost: The 3D U-Net upsampler adds computational overhead, which might be a factor for real-time deployment in high-throughput clinical settings.

Summary updated

Abstract

Accurate localization of abnormalities in three-dimensional (3D) CT scans is essential for linking automated predictions with clinically relevant pathological findings. We propose Uni-Loc, a vision-language framework that unifies pathology classification and 3D localization using scan-level supervision. Uni-Loc derives both predictions and localization maps from spatial similarities between pathology-conditioned queries and image features. A token-to-voxel upsampling module enables high-resolution localization, while gradient-guided thresholding identifies pathology regions without additional learned parameters. Evaluated on five localization tasks comprising 355 annotated CT scans, Uni-Loc achieves macro-averaged voxel-level AUROC/AUPR of 96.1\%/34.5\%, outperforming relevance propagation (80.7%/13.9%) and gradient saliency (88.7%/8.1%). Uni-Loc also achieves classification AUROC of 88.39% on CT-RATE and 78.03% on RAD-ChestCT, improving over a matched baseline. These results demonstrate effective 3D pathology localization without requiring pathology-specific voxel-level supervision.
BibTeX
@misc{lee2026uni,
  author = {Hao-Chih Lee and Zahi A. Fayad and Xueyan Mei},
  title = {{Uni-Loc: Weakly Supervised 3D CT Pathology Localization via Spatially Grounded Vision-Language Classification}},
  year = {2026},
  eprint = {2611.13330},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.13330},
}
Figure