Abstract
Deep learning models demonstrate remarkable efficacy in detecting malignancy within Whole Slide Images (WSIs), yet their "black box" nature limits clinical trust. We propose a novel explainability framework that utilizes Non-negative Matrix Factorization (NMF) to cluster internal activations of Convolutional Neural Networks (CNNs). Unlike traditional saliency maps that merely highlight regions of interest, our method generates global, semantically meaningful tissue segmentations. These clusters align with specific morphological features recognized by pathologists, bridging the gap between automated classification and human-interpretable diagnostics.
Introduction: The Interpretability Gap
While deep learning has revolutionized digital pathology, a significant "trust gap" persists. Pathologists require more than binary classifications; they need to understand the morphological reasoning behind a model's decision. Existing eXplainable AI (XAI) techniques, such as GradCAM, produce one-dimensional saliency maps. While these indicate where a model is looking, they fail to describe what morphological features—such as gland structure or cell density—are driving the prediction.
Methodology
The proposed approach interprets a VGG-16 model trained on prostate cancer detection through three primary steps:
- Activation Extraction: Feature vectors are extracted from the final convolutional layer of the trained model.
- NMF Clustering: Utilizing the non-negative nature of ReLU activations, the authors use Non-negative Matrix Factorization (NMF) to decompose these vectors into $K$ distinct classes. This allows for the visualization of "class intensity" across spatial locations.
- Global Visualization: The resulting clusters are overlaid on the original H&E stained tissue, creating a semantic map based purely on the model's learned representations.
Results and Discussion
The study reveals that CNNs naturally learn to distinguish between various tissue types (e.g., stroma vs. epithelium) even when only trained for binary cancer classification.
Pathological Alignment
A resident pathologist confirmed that the generated clusters correspond to distinct histological structures. For instance, specific clusters were identified as "fascicular structures with sparse elongated nuclei" or "chains of densely packed nuclei," providing a level of granularity absent in standard heatmaps.
Quantitative Validation
To ensure these clusters captured the model's decision-making logic, the researchers trained a linear proxy model using only cluster weights. This proxy achieved an AUC of 0.998, indicating that the NMF clusters retain nearly all the predictive information used by the original "black box" CNN. The analysis identified specific clusters (Classes 2, 4, and 5) that strongly correlate with cancerous regions, while others represent benign or background tissue.
Conclusion
By moving from simple heatmaps to structured clustering, this method provides a transparent view into the internal logic of pathology models. This approach not only identifies where a model is looking but provides a semantically grounded explanation of what it sees. Future work will focus on scaling this approach to modern transformer architectures and evaluating its impact on clinical workflows.
Summary updated