Introduction
Medical imaging modalities such as Whole-Slide Images (WSIs) and CT scans present a significant computational challenge due to their massive scale—often containing billions of pixels or hundreds of slices. To process these, researchers typically employ "divide-and-conquer" pipelines that extract small patch embeddings for global prediction. However, these aggregators often suffer from a loss of spatial context and rely on noisy post-hoc visualization tools like Grad-CAM, which can miss clinically vital details and produce blurry results.
The INSIGHT Framework
The INSIGHT (Integrated Network for Segmentation and Interpretation with Generalized Heatmap Transmission) architecture addresses these issues by building interpretability directly into the model. Unlike traditional methods that require expensive pixel-level annotations for segmentation, INSIGHT operates under a weakly-supervised paradigm, using only image-level labels to generate high-quality segmentation heatmaps.
Key architectural innovations include:
- Dual-Module Architecture: A Detection Module captures fine-grained textures, while a Context Module uses larger kernels to capture broader spatial relationships and suppress false positives in irrelevant regions.
- Heatmap Fusion & Dynamic Thresholding: Module outputs are fused into an internal heatmap, which is then sharpened using Otsu’s algorithm to filter background noise.
- SmoothMax Pooling: By considering all relevant activations rather than just the single most positive patch, SmoothMax allows the model to learn more accurate lesion boundaries and account for the entire area of a lesion.
Key Results and Insights
INSIGHT was evaluated on three major benchmarks: MosMed (CT), CAMELYON16 (WSI), and BRACS (WSI). The framework achieved state-of-the-art performance in both classification and segmentation:
- Classification: Reached a 96.2% AUC on the MosMed CT dataset, nearly 5% higher than the best baseline.
- Segmentation: Achieved a 74.6% Dice score on CAMELYON16, outperforming competing models by 6.9%.
A critical finding is that forcing the model to generate heatmaps during the forward pass acts as a regularizer, improving classification accuracy by preventing the model from focusing on background artifacts. Furthermore, INSIGHT demonstrates a superior ability to detect small lesions that are often "blurred away" by standard Multiple Instance Learning (MIL) methods.
Conclusion
INSIGHT represents a shift toward "explainable-by-design" architectures in medical AI. By maintaining spatial resolution and accounting for the entire lesion area, it provides high-fidelity clinical insights without the need for dense annotations. Future work will explore its application to 3D volumetric data and integration with emerging foundation models.
Summary updated