TL;DR
This paper introduces a novel method for compressing deep learning models used in medical imaging by leveraging interpretability techniques. By identifying and removing less important neurons based on their contribution to model predictions, they achieve significant model compression with minimal loss in accuracy, making these powerful AI tools more accessible for real-world clinical applications.
What Problem Does This Paper Address?
Deep learning has revolutionized medical image analysis, achieving impressive accuracy in tasks like disease detection and segmentation. However, the widespread adoption of these models in clinical settings is hindered by two major challenges: their substantial computational requirements (making deployment difficult in resource-constrained environments) and their "black-box" nature, which erodes trust among medical professionals who need to understand why a model makes a particular decision. Model compression techniques, particularly pruning, aim to address the computational burden, but traditional methods often lack a direct link to the model's reasoning, potentially removing crucial components. This paper tackles the problem of creating efficient and interpretable medical imaging models by integrating interpretability directly into the pruning process.
Methods
The core of this work is an "interpretability-aware pruning" framework. Instead of relying on traditional heuristics like weight magnitude for pruning, this method uses interpretability techniques to guide the process. Specifically, they employ three attribution methods:
- Layer-wise Relevance Propagation (LRP): Propagates prediction "relevance scores" backward through the network to identify contributions of individual neurons.
- DL-Backtrace (DLB): A model-agnostic method that assigns relevance scores across layers to reveal feature importance and information flow.
- Integrated Gradients (IG): A path-based technique that quantifies the contribution of input features to a model's prediction by integrating gradients.
These methods are used to compute importance scores for individual neurons. These scores are then aggregated to get a per-neuron importance. This information is used to selectively retain only the most relevant parts of each layer, enabling targeted compression. The paper also explores different sampling strategies (Clustering-based, Confidence-based, Random) for computing these importance scores to ensure robustness.
Datasets
The authors evaluated their framework on four diverse medical imaging datasets:
- MURA Dataset: Musculoskeletal radiographs (normal vs. abnormal).
- KVASIR Dataset: Endoscopic images of the gastrointestinal tract.
- CPN Dataset: Lower limb images for diagnosing common peroneal nerve injuries.
- FETAL Planes Dataset: High-resolution ultrasound images of fetal heads.
They trained and evaluated their models on these datasets using three popular architectures: VGG19, ResNet50, and Vision Transformer (ViT).
Key Insights and Ideas
- Interpretability as a Pruning Compass: The most significant insight is that interpretability techniques can effectively guide model pruning, moving beyond simple heuristics to focus on the functional importance of model components. This leads to more efficient and transparent models.
- Lossless Compression is Achievable: The framework demonstrates the ability to achieve substantial compression rates with minimal loss in accuracy, sometimes even achieving "lossless pruning" where accuracy is maintained despite significant parameter reduction.
- ViTs are Remarkably Robust: Vision Transformers (ViTs) showed exceptional resilience to pruning, maintaining performance even at very high pruning rates (up to 80-85%), suggesting they have inherent redundancy or feature richness that makes them well-suited for this approach.
- Clustering-based Sampling is Key: Clustering-based sampling emerged as the most effective strategy for selecting representative samples to compute importance scores, leading to more robust pruning results compared to random or confidence-based methods.
- Pruning Can Sometimes Improve Accuracy: In certain scenarios, pruning actually led to a slight increase in accuracy. This counterintuitive finding suggests that interpretability-guided pruning can effectively remove redundant or noisy neurons that might otherwise negatively influence predictions, leading to better generalization.
Key Results
- The proposed interpretability-aware pruning method consistently enables high model compression (up to 80-85%) with accuracy drops within a 5% margin, even at moderate pruning rates.
- ViT-based models demonstrated the highest robustness, tolerating up to 85% pruning with minimal degradation.
- VGG architectures could tolerate up to 50% pruning, while ResNets were more sensitive, typically stabilizing up to 30% pruning.
- Layer-wise Relevance Propagation (LRP) consistently enabled the highest pruning thresholds while maintaining accuracy, followed by DL-Backtrace (DLB) and Integrated Gradients (IG).
- Clustering-based sampling outperformed other strategies, particularly for CNN architectures on CPN and MURA datasets.
Limitations and Future Work
- Further Empirical Validation: While promising, the authors acknowledge that further empirical validation is needed, especially for real-world clinical deployment.
- Hypothesis on Domain Shift: They hypothesize that pruning pre-trained models using region-specific, representative data could improve generalization and mitigate domain shift issues in deployment.
- Potential for Enhanced Generalization: The observed accuracy improvements after pruning suggest that this method could be a pathway to building not only lightweight and interpretable AI systems but also potentially more robust ones.
Summary updated