DepViT-CAD: Deployable Vision Transformer-Based Cancer Diagnosis in Histopathology

Ashkan Shakarami, Lorenzo Nicolè, Rocco Cappellesso, Angelo Paolo Dei Tos, Stefano Ghidoni

TL;DR

DepViT-CAD is a new, deployable AI framework that uses a custom "Multi-Attention Vision Transformer" (MAViT) to diagnose 10 major cancer types with over 94% sensitivity. By combining the efficiency of convolutional neural networks with the global context of transformers, it provides pathologists with a fast, interpretable tool for navigating complex clinical cases.

What Problem Does This Paper Address?

The global incidence of cancer is rising, but the number of qualified pathologists isn't keeping pace. This "diagnostic gap" leads to heavy workloads and potential delays in treatment. While digital pathology—the digitization of glass tissue slides into Whole-Slide Images (WSIs)—has opened the door for AI assistance, most existing models struggle with two things:

  1. Fine-grained differentiation: It is relatively easy for an AI to tell a brain tumor from a skin tumor. It is much harder to distinguish between subtle subtypes (like different forms of glioma or lung cancer) that look nearly identical under standard H&E (Hematoxylin and Eosin) staining.
  2. Deployability: Many high-performing models are computationally "heavy" or "black boxes" that lack the transparency needed for clinical trust.

Methods

The authors introduce MAViT (Multi-Attention Vision Transformer), which uses a "best of both worlds" architecture:

  • The Backbone: It starts with EfficientNet-B3, a standard convolutional neural network (CNN) that is excellent at picking up local textures and cellular details.
  • The Transformer (VTM): To understand the broader "geography" of the tissue, the model uses a Vision Transformer Module. Crucially, they implemented Linear Attention (inspired by the Linformer). Standard transformers get exponentially slower as image size increases; linear attention keeps the computational cost manageable for massive pathology slides.
  • Dual Fusion Strategy (DFS): The model doesn't just look at the final output; it merges features from different depths of the network. This ensures the AI considers both tiny cellular anomalies and large-scale tissue structures simultaneously.
  • Interpretability Tools: To help doctors trust the results, the system generates D-Graphs (a quadrant-based visualization of accuracy), probability heatmaps, and label maps that show exactly where the AI thinks the tumor is located.

Datasets

The study utilized a robust mix of research and "real-world" data:

  • TgCancer-ds: A large-scale dataset derived from The Cancer Genome Atlas (TCGA), containing 1,283 WSIs across 10 cancer types and non-tumor tissue.
  • CliniR-ds: A "real-world" external validation set of 50 routine clinical cases from pathology labs in Italy. This included both surgical specimens and biopsies, which are often harder for AI to process due to their smaller size and fragmented nature.

Key Insights and Ideas

  • Linear Attention is the "Enabler": The move from quadratic to linear complexity in the attention mechanism is what makes this "deployable." It allows the model to capture global context without requiring a supercomputer.
  • Context Matters More than Just Pixels: The ablation study showed that adding the Transformer module (VTM) and the Fusion strategy (DFS) significantly boosted sensitivity. A pure CNN might see the cells, but the Transformer understands the neighborhood.
  • Tumor-Centric, Not Organ-Centric: The model was trained to identify tumor-specific features rather than just memorizing what a "lung" or "brain" looks like. This was proven by its ability to correctly identify non-tumor (healthy) regions even within a slide dominated by cancer.

Key Results

  • High Sensitivity: The system achieved a diagnostic sensitivity of 94.11% on the TCGA test set and 92% on the real-world clinical cohort.
  • Superior Accuracy: MAViT reached a patch-level accuracy of 98.12%, outperforming standard models like ResNet50 and MobileNetV3.
  • Speed: Inference time is roughly 0.0183 seconds per image patch, making it fast enough for integration into actual clinical workflows.
  • Clinical Utility: In one specific case of a brain tumor (glioma) where human review was ambiguous, DepViT-CAD correctly predicted the subtype within minutes—a result later confirmed by expensive molecular testing.

Limitations and Future Work

Despite its success, the authors note a few hurdles:

  • Glioma Confusion: The model occasionally confused similar brain tumor subtypes (e.g., Astrocytoma vs. Oligodendroglioma). These are notoriously difficult even for expert humans and often require molecular analysis to settle.
  • Biopsy Representation: The training set was primarily composed of surgical specimens. While the model performed well on biopsies in testing, the authors want to include more biopsy-specific data in the future to improve generalization.
  • Soft-Label Aggregation: Currently, the model uses "majority voting" to reach a final diagnosis. The authors suggest that "soft-label" aggregation (averaging probabilities) might handle ambiguous, heterogeneous slides even better in future versions.

Summary updated

Abstract

Accurate and timely cancer diagnosis from histopathological slides is vital for effective clinical decision-making. This paper introduces DepViT-CAD, a deployable AI system for multi-class cancer diagnosis in histopathology. At its core is MAViT, a novel Multi-Attention Vision Transformer designed to capture fine-grained morphological patterns across diverse tumor types. MAViT was trained on expert-annotated patches from 1008 whole-slide images, covering 11 diagnostic categories, including 10 major cancers and non-tumor tissue. DepViT-CAD was validated on two independent cohorts: 275 WSIs from The Cancer Genome Atlas and 50 routine clinical cases from pathology labs, achieving diagnostic sensitivities of 94.11% and 92%, respectively. By combining state-of-the-art transformer architecture with large-scale real-world validation, DepViT-CAD offers a robust and scalable approach for AI-assisted cancer diagnostics. To support transparency and reproducibility, software and code will be made publicly available at GitHub.
BibTeX
@misc{shakarami2025depvit,
  author = {Ashkan Shakarami and Lorenzo Nicolè and Rocco Cappellesso and Angelo Paolo Dei Tos and Stefano Ghidoni},
  title = {{DepViT-CAD: Deployable Vision Transformer-Based Cancer Diagnosis in Histopathology}},
  year = {2025},
  eprint = {2507.10250},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.10250},
}
Figure