Abstract
With the increasing demands for understanding the internal behaviors of deep networks, Explainable AI (XAI) has made remarkable progress in interpreting model decisions. A family of attribution techniques has been proposed, highlighting whether input pixels are responsible for a model's prediction. However, existing attribution methods often suffer from a lack of rule guidance and require further human interpretation. In this paper, we construct 'if-then' logic rules that are sufficiently precise locally. Moreover, a novel rule-guided method, dynamic ablation (DA), is proposed to find a minimal bound sufficient in an input image to justify the network's prediction and aggregate iteratively to reach a complete attribution. Both qualitative and quantitative experiments are conducted to evaluate the proposed DA. We demonstrate the advantages of our method in providing clear and explicit explanations that are also easy for human experts to understand. Besides, through the attribution on a series of trained networks with different architectures, we show that more complex networks require less information to make a specific prediction.
Introduction
Deep Neural Networks (CNNs) are notoriously "black boxes." To understand their internal logic, researchers use attribution methods—techniques that produce saliency maps to show which pixels influenced a prediction. However, existing methods like Grad-CAM or RISE often lack formal guidance. They might highlight large regions that contribute to a score without identifying the minimal sufficient information the model needs. For instance, if a model identifies a "dog," does it need the whole animal or just the ears? Current methods often fail to distinguish between "helpful" pixels and "necessary" pixels, leading to blurry explanations that require significant human interpretation.
Methods: Logic Rules and Dynamic Ablation
We introduce a framework called Logic Rule Guided Attribution powered by the Dynamic Ablation (DA) algorithm. This approach defines two types of local rules:
- Minimal-Positive Rule (Pos-R): "If parts $p_1...p_k$ are present, then the image is classified as $y$." This seeks the smallest patch that maintains the original prediction.
- Minimal-Negative Rule (Neg-R): "If parts $p_1...p_k$ are absent, then the image is not classified as $y$." This identifies the most critical features for a classification.
To find these parts, DA uses $k$ "dynamic circles" that iteratively shrink and move across the image. If the model maintains its prediction after a shrink, the change is kept. To overcome the computational cost of pixel-level ablation, we utilize SLIC superpixels to group similar pixels. This optimization allows DA to generate an explanation in approximately 23.8 seconds, compared to over 20 minutes for raw pixel-based methods.
Experimental Results
Our method was evaluated using ImageNet2012 and CIFAR-100 benchmarks across multiple architectures, including AlexNet, VGG, and ResNet.
Qualitative Analysis
As shown in Figure 5, DA provides much more localized and interpretable attributions than baseline methods. A key observation is that models often rely on "distractor" features or reasoning shortcuts. For example, when classifying a "miniskirt," the model focuses almost entirely on the person's legs rather than the garment itself. DA makes these shortcuts explicit, whereas methods like Grad-CAM provide a more generalized heatmap.
Quantitative Performance
In the "Insertion Game"—where pixels are added back to a blank image based on their importance—DA achieved higher accuracy with fewer pixels than state-of-the-art methods like RISE and Extremal Perturbation (EP). This confirms that our logic-rule approach identifies more relevant features for the model's decision process.
Complexity vs. Information Efficiency
One of the most striking findings is the relationship between model complexity and the Minimum Attribution Ratio (MAR)—the percentage of the image the model actually needs to see to be confident. We discovered that more complex networks are more efficient observers:
- AlexNet: 11.68% MAR
- VGG16: 4.02% MAR
- ResNet50: 3.50% MAR
This suggests that deeper, more complex architectures are better at distilling high-level concepts from tiny fragments of data.
Conclusion and Future Work
Dynamic Ablation provides a robust framework for generating precise, rule-based explanations for deep learning models. While the current implementation relies on circular masks, which may not perfectly capture irregular shapes, it offers a significant step toward transparent AI. Future work will focus on applying these logic rules to interpret hidden neurons and layers to understand how concepts are constructed internally.
Summary updated