TL;DR
This paper introduces "Route, Interpret, Repeat," a novel framework that iteratively decomposes a high-performing blackbox model into a Mixture of Interpretable Experts (MoIE). By carving out symbolic logic explanations for specific data subsets while routing "harder" samples to a residual blackbox, the authors achieve state-of-the-art performance without sacrificing the transparency of inherently interpretable models.
What Problem Does This Paper Address?
In machine learning, there is a long-standing dichotomy: you can either use a flexible, high-performing Blackbox (like a deep neural network) and try to explain it after the fact (post hoc), or you can build an Interpretable-by-Design model from the start.
Both have major drawbacks. Post hoc explanations often lack fidelity to the original model and can be prone to confirmation bias. Conversely, interpretable-by-design models (like Concept Bottleneck Models) often underperform because they assume a single, simple function can explain an entire complex dataset. This paper bridges the gap, aiming to retain blackbox-level accuracy while providing the mechanistic, actionable explanations typical of symbolic models.
Methods
The authors propose an iterative "divide-and-conquer" approach called Route, Interpret, Repeat. Instead of trying to explain the whole model at once, they break it down into a sequence of experts.
- Route: A learnable gating mechanism (a "selector") examines an input and decides if it can be explained by the current interpretable expert.
- Interpret: If selected, the sample is processed by a Neuro-Symbolic Interpretable Expert. This expert uses First Order Logic (FOL) to provide reasoning based on human-understandable concepts (e.g., "If the bird has a black forehead AND grey legs, then it is a Bay Breasted Warbler").
- Repeat: Samples that the current expert cannot explain are routed to a Residual Blackbox. This residual is then used as the starting point for the next iteration, where a new expert is "carved out."
This process continues until a desired proportion of the data is covered by interpretable experts. The final model is a Mixture of Interpretable Experts (MoIE) plus a small residual for the most "unexplainable" samples.
Datasets
The framework was evaluated across diverse vision and medical imaging tasks:
- CUB-200 & AwA2: Standard benchmarks for fine-grained bird and animal classification.
- HAM10000 & SIIM-ISIC: Medical datasets for skin lesion and melanoma classification.
- MIMIC-CXR: A large-scale chest x-ray dataset used here for identifying pleural effusion.
- Waterbirds: A synthetic dataset specifically designed to test "shortcut learning" (e.g., a model mistakenly identifying a bird based on the background rather than the bird itself).
Key Insights and Ideas
- Blackboxes are Multi-Faceted: The core hypothesis is that a single blackbox actually encodes multiple different interpretable strategies. MoIE succeeds by identifying these distinct strategies and assigning them to specialized experts.
- Instance-Specific Explanations: Unlike standard models that give a generic explanation for a whole class, MoIE identifies diverse, instance-specific concepts. For example, one expert might identify a bird by its wing color, while another focuses on its beak shape.
- Identifying "Hard" Samples: The iterative process naturally sorts data by difficulty. Samples that reach the final residuals are quantitatively shown to be the "hardest" for the original blackbox to classify correctly.
- Fixing Shortcuts with Logic: Because the experts use First Order Logic, humans can intervene. If an expert is found to be using a "shortcut" (like a forest background to identify a land bird), the logic can be modified or the bias can be removed using metadata normalization.
Key Results
- Performance Parity: MoIE achieves accuracy nearly identical to the original blackbox models, significantly outperforming traditional interpretable-by-design baselines like CBMs. For instance, on CUB-200 (VIT), MoIE reached 91% accuracy compared to the blackbox's 92%, while the CBM baseline lagged at 86%.
- Higher Fidelity: MoIE-identified concepts attained roughly 20% higher completeness scores than post-hoc baselines, meaning the concepts chosen were much more essential to the model's actual decision-making process.
- Effective Intervention: In "test-time interventions"—where a human corrects a misidentified concept—MoIE showed superior recovery. On CUB-200, intervening on just 20 concepts boosted MoIE accuracy from 91% to 96%.
- Shortcut Removal: The authors successfully used MoIE to identify and eliminate spurious background biases in the Waterbirds dataset, creating a more robust model that generalizes better across different environments.
Limitations and Future Work
- Causality: The authors acknowledge that while the identified concepts are highly predictive, they are not necessarily causal. Future work will focus on estimating inter-concept interactions to better assess causal effects.
- Modality Expansion: The current study focuses on images. The researchers aim to apply the "Route, Interpret, Repeat" framework to other data types, such as text and video, in the future.
- Computational Overhead: While experts are shallow and efficient, the iterative training process adds some complexity compared to training a single standard model.
Summary updated