Hybrid-View Attention Network for Clinically Significant Prostate Cancer Classification in Transrectal Ultrasound

Zetian Feng, Juan Fu, Xuebin Zou, H. Ye, Hong Wu, Jianhua Zhou, Yi Wang

TL;DR

Researchers have developed the Hybrid-View Attention (HVA) network, a novel deep learning architecture that improves the detection of clinically significant prostate cancer (csPCa) in 3D transrectal ultrasound (TRUS). By simultaneously analyzing transverse and sagittal views using a CNN-Transformer hybrid, the model compensates for the low contrast and "blurry" resolution typical of ultrasound, significantly outperforming traditional single-view diagnostic methods.

What Problem Does This Paper Address?

Prostate cancer is a leading cause of death among men, and distinguishing between aggressive (clinically significant) and indolent (insignificant) cases is vital for deciding on treatment. While Transrectal Ultrasound (TRUS) is the standard tool for guiding biopsies due to its low cost and real-time nature, it has two major flaws:

  1. Low Contrast: It is often difficult to see the boundaries of a tumor.
  2. Anisotropic Resolution: Ultrasound images are sharp in the imaging plane but "blurry" (low resolution) along the scanning direction due to slice thickness.

Most existing AI models only look at the transverse view, ignoring the sagittal view. This is like trying to understand a 3D object by only looking at it from the front, never the side. This paper aims to bridge that gap by leveraging the complementary information hidden in these two orthogonal perspectives.

Methods

The authors propose a dual-stream encoder architecture that combines the strengths of Convolutional Neural Networks (CNNs) and Transformers:

  • CNN-Transformer Hybrid: CNN layers (Residual Blocks) are used first to extract fine-grained local textures, while Transformer-based modules handle "global dependencies"—understanding how different parts of the prostate relate to each other across the entire volume.
  • Intra-View Attention (IVA): This module refines features within a single view (e.g., just the transverse plane) by capturing long-range spatial and channel relationships.
  • Cross-View Attention (CVA): This is the core innovation. It uses features from one view (e.g., sagittal) as "queries" to look for relevant information in the other view (transverse). This allows the model to "fill in the blanks" caused by the low resolution in the scanning direction of a single view.
  • Hybrid-View Adaptive Fusion (HVAF): Instead of just adding the two views together, this module dynamically re-weights features. It uses channel and spatial attention to decide which parts of which view are most reliable for the final classification.

Datasets

The study utilized an in-house 3D TRUS dataset from the Cancer Center of Sun Yat-sen University:

  • Size: 590 subjects (590 pairs of transverse and sagittal volumes).
  • Labels: 401 cases of clinically significant PCa (csPCa) and 189 cases of clinically insignificant PCa (cisPCa), all confirmed via biopsy and whole-gland prostatectomy.
  • Pre-processing: All volumes were resized to a standard $128 \times 128 \times 128$ cube.

Key Insights and Ideas

  • Orthogonal Synergy: The researchers realized that the "blurriness" in the transverse view's depth is exactly where the sagittal view is sharpest. By using Cross-View Attention, the model effectively "borrows" resolution from the orthogonal plane.
  • Efficiency via Shared Queries: To keep the computational cost down, the IVA module uses a shared query-key mechanism, allowing the model to process spatial and channel information in parallel without a massive increase in parameters.
  • Beyond Simple Concatenation: The ablation studies showed that simply "stacking" the two views (concatenation) provides a boost, but the Adaptive Fusion (HVAF) provides a much larger jump in performance because it learns where to look in each view.

Key Results

The HVA network demonstrated superior performance across all major metrics compared to single-view models and existing multi-view fusion techniques:

  • AUC (Area Under Curve): Achieved 0.8534, compared to 0.7562 for a standard transverse-only model.
  • F1-Score: Reached 0.8611.
  • Accuracy: 80.26%.
  • Comparison: The model outperformed other state-of-the-art multi-view methods (like DMVFN and MMFN) by a margin of roughly 6-9% in AUC.

Qualitative results using Class Activation Mapping (CAM) showed that the model's "attention" closely aligned with the actual locations of tumors identified by pathologists.

Limitations and Future Work

  • Single-Center Data: The dataset, while large for this field, comes from a single institution. The authors note that external validation on data from different ultrasound machines and hospitals is necessary to prove the model's robustness.
  • Resolution Limits: While the model compensates for anisotropic resolution, it is still limited by the inherent low contrast of B-mode ultrasound.
  • Future Directions: The authors suggest that integrating this "hybrid-view" approach with other modalities, such as MRI (multi-modal fusion), could further enhance diagnostic precision.

Check out the code: The authors have made their implementation available on GitHub at https://github.com/mock1ngbrd/HVAN.

Summary updated

Abstract

Prostate cancer (PCa) is a leading cause of cancer-related mortality in men, and accurate identification of clinically significant PCa (csPCa) is critical for timely intervention. Transrectal ultrasound (TRUS) is widely used for prostate biopsy; however, its low contrast and anisotropic spatial resolution pose diagnostic challenges. To address these limitations, we propose a novel hybrid-view attention (HVA) network for csPCa classification in 3D TRUS that leverages complementary information from transverse and sagittal views. Our approach integrates a CNN-transformer hybrid architecture, where convolutional layers extract fine-grained local features and transformer-based HVA models global dependencies. Specifically, the HVA comprises intra-view attention to refine features within a single view and cross-view attention to incorporate complementary information across views. Furthermore, a hybrid-view adaptive fusion module dynamically aggregates features along both channel and spatial dimensions, enhancing the overall representation. Experiments are conducted on an in-house dataset containing 590 subjects who underwent prostate biopsy. Comparative and ablation results prove the efficacy of our method. The code is available at https://github.com/mock1ngbrd/HVAN.
BibTeX
@misc{feng2025hybrid,
  author = {Zetian Feng and Juan Fu and Xuebin Zou and H. Ye and Hong Wu and Jianhua Zhou and Yi Wang},
  title = {{Hybrid-View Attention Network for Clinically Significant Prostate Cancer Classification in Transrectal Ultrasound}},
  year = {2025},
  eprint = {2507.03421},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.03421},
}
Figure