Introduction: The Interpretability Gap
In the field of Explainable AI (XAI), researchers frequently rely on "attribution maps"—vibrant heatmaps designed to highlight the pixels a model deems most important for a given classification. However, these visualizations often fall into the "eye candy" trap: a heatmap that looks intuitive to a human observer does not necessarily reflect the model's actual internal logic. Furthermore, there has historically been a significant measurement gap; validating whether an AI's strategy aligns with biological vision (human or non-human primate) has required labor-intensive psychophysical methods like "Bubbles," which can demand tens of thousands of trials for even a small set of images.
To address these challenges, researchers have introduced MAPS (Masked Attribution-based Probing of Strategies), a computational framework designed to adjudicate between competing XAI methods by grounding them in behavioral data.
The MAPS Framework: Causal Testing of AI Strategies
The core innovation of MAPS is the Explanation-Masked Image (EMI). Rather than simply looking at a heatmap, the framework uses the attribution map from a "Reference" model (e.g., an Artificial Neural Network or ANN) to create a mask. By applying a percentile cutoff—keeping only the top 5% or 10% of pixels identified as "important"—researchers generate a minimal image that is then shown to a "Target" system (a human, monkey, or another AI).
This approach shifts XAI from qualitative visualization to causal testing. If the Target system maintains high accuracy and exhibits the same image-by-image error patterns (consistency) as the Reference model when viewing these sparse EMIs, the explanation is considered "aligned." This allows for a principled way to determine which interpretability tools truly capture the "how" of visual recognition.
Key Insights: Method Choice and Biological Alignment
The study, which utilized MS COCO images for a 10-way classification task, yielded several critical findings:
- Method Choice Trumps Training: The specific XAI algorithm used (e.g., Saliency vs. Integrated Gradients) causes more variation in explanations than the random initialization of the network itself. This underscores the necessity of choosing the right interpretability tool for the task.
- Noise Tunnel Saliency as the Gold Standard: Among 12 tested methods, Noise Tunnel Saliency consistently produced the most human- and monkey-aligned explanations. In contrast, older methods like Deconvolution failed to capture biological visual strategies.
- Neural Grounding: The model-method combinations that showed the best behavioral alignment also best predicted the firing patterns of neurons in the primate inferior temporal (IT) cortex, suggesting that MAPS-derived strategies reflect the representational logic of the ventral visual stream.
Validating Against the "Gold Standard"
MAPS was benchmarked against the traditional "Bubbles" method, which randomly masks images to identify diagnostic regions. While Bubbles is highly accurate, it is inefficient, requiring roughly 36,000 trials to map 40 images. MAPS achieved the same behavioral validity using significantly fewer trials (~4,000), making it a scalable alternative for large-scale strategy probing.
Conclusion and Future Directions
MAPS provides a robust, behaviorally grounded framework for aligning AI explanations with biological vision. While currently focused on static images, future work aims to extend MAPS to dynamic video and temporally evolving decisions. By bridging the gap between colorful heatmaps and actual visual strategies, MAPS moves the field closer to AI systems that not only perform like humans but also "see" like them.
Summary updated