Abstract
Explainable AI (XAI) often struggles to bridge the gap between low-level input features and high-level human concepts. This paper introduces a rigorous statistical framework to determine which semantic concepts (e.g., "stripes," "wings") are truly influential in a black-box model's predictions. By framing importance as a sequential "game of betting," the authors provide a method to rank concepts with formal guarantees on false positives, bypassing the need for simplified surrogate models.
Introduction
Most XAI tools focus on the input space, highlighting pixels or words. However, humans reason using concepts—a radiologist looks for "spiculation," not just pixel clusters. While Concept Bottleneck Models (CBMs) address this, they often rely on linear surrogates that may not faithfully represent the original model. This work formalizes semantic importance through the lens of conditional independence testing, providing precise statistical guarantees.
Methodology
The authors propose testing importance at three granularities:
- Global Importance (SKIT): Population-wide dependence.
- Global Conditional Importance (c-SKIT): Unique information not captured by other concepts.
- Local Conditional Importance (x-SKIT): Importance for a specific individual prediction.
Testing by Betting
Instead of traditional p-values, the framework uses Testing by Betting. A "bettor" wagers against a null hypothesis (e.g., "this concept is not important"). If the concept is important, the bettor accumulates wealth (an e-value). Rejection occurs when wealth crosses a threshold ($1/\alpha$). This sequential approach is data-efficient and allows for ranking concepts based on rejection time.
Technical Implementation
The framework utilizes Maximum Mean Discrepancy (MMD) to measure distances between probability distributions. It includes a greedy post-processor to control the False Discovery Rate (FDR) when testing multiple concepts simultaneously.
Experimental Evaluation
The framework was tested on synthetic data, Counting MNIST, and real-world datasets like Animals with Attributes 2 (AwA2) and CUB-200-2011.
Key Results
- High Fidelity: On AwA2, the framework achieved 99.5% accuracy in identifying important concepts, significantly outperforming standard Concept Bottleneck Models (~95%).
- Model Agnosticism: Importance ranks were stable across different vision-language models (CLIP, OpenClip, BLIP), suggesting shared semantic strategies.
- Local Insights: The
x-SKITtest successfully identified concepts relevant to specific images (e.g., "fishing" for a dog in water) that weren't globally relevant.
Discussion and Limitations
The framework's power depends on the quality of conditional samplers and kernel selection. While non-parametric KDE-based samplers were used, high-dimensional spaces remain a challenge. Additionally, the method tests the provided concepts; it does not discover new ones.
Conclusion
This betting-based approach offers a robust, statistically grounded way to interpret black-box models through human-understandable concepts. By avoiding surrogates and providing formal guarantees, it sets a new standard for reliable semantic explanations in AI.
Summary updated