Abstract
Sokol et al., (2022). What and How of Machine Learning Transparency: Building Bespoke Explainability Tools with Interoperable Algorithmic Components. Journal of Open Source Education, 5(58), 175, https://doi.org/10.21105/jose.00175
Introduction
As machine learning models grow in complexity, the tools designed to explain them—Explainable AI (XAI)—have paradoxically become "black boxes" themselves. Popular methods like LIME (Local Interpretable Model-agnostic Explanations) or SHAP (SHapley Additive exPlanations) are frequently applied as monolithic, one-size-fits-all solutions. However, using these tools without a deep understanding of their internal mechanics can result in misleading explanations or poor performance on specialized datasets.
This paper addresses a critical gap in XAI education and implementation: the lack of hands-on resources that treat explainability as a modular engineering task. The authors propose a "glass-box" approach, decomposing surrogate explainers into interchangeable, tunable algorithmic components.
The Modular Surrogate Explainer Framework
The core of the proposed methodology is the concept of Modular Surrogate Explainers, particularly for tabular data. A surrogate explainer functions by training a simple, inherently interpretable model (such as a decision tree) to approximate the behavior of a complex "black-box" model within a specific local neighborhood.
The authors introduce the bLIMEy (build LIME yourself) meta-algorithm. This framework breaks down the explanation process into three distinct, interoperable building blocks:
- Interpretable Representation Composition: This step involves transforming raw, high-dimensional data into formats that humans can easily grasp, such as using quartile discretization or tree-based partitioning.
- Data Sampling: To understand how a black-box model behaves, synthetic data points are generated around the instance being explained. Techniques like Gaussian sampling or "mixup" are employed to probe the model's decision boundaries.
- Explanation Generation: This final stage involves feature selection, sample weighting, and training the surrogate model (e.g., a Linear Model or Decision Tree) to produce the human-readable explanation.
Educational Resources and FAT Forensics
To facilitate the practical application of these concepts, the authors leverage FAT Forensics, an open-source Python package. This library is specifically designed to evaluate the fairness, accountability, and transparency of machine learning modules.
Unlike traditional static teaching methods, this work provides a comprehensive educational ecosystem including Jupyter Notebooks, interactive widgets, and "no-code" experimentation tools. These resources allow practitioners to "bring their own data," enabling the creation of bespoke explainers tailored to specific domains like finance or medicine. A key innovation presented is the publishing workflow itself: online documents, slides, and computational notebooks are all generated from a single, unified source, ensuring consistency across learning formats.
Key Insights
- XAI as a Pipeline: Explainability tools should be viewed as sequences of independent algorithmic steps. This perspective allows developers to swap out underperforming modules (like a poor sampling method) without discarding the entire tool.
- The Necessity of Bespoke Tools: There is no universal explainer. A strategy effective for one dataset may fail for another. Tuning individual modules is essential for generating trustworthy and meaningful explanations.
- Interactivity and Evaluation: Moving from static slides to interactive notebooks significantly improves engagement and the understanding of algorithmic failure modes. Furthermore, the paper stresses that explanations cannot be trusted without quantitative metrics to verify the "faithfulness" and "robustness" of the surrogate model.
Conclusion and Future Directions
The primary contribution of this work is the democratization of XAI through open-source education. The materials have already seen success at major conferences like the 2020 European Conference on Machine Learning (ECML-PKDD) and in various doctoral and undergraduate settings.
While the current focus is on tabular data, future work aims to extend these modular tools to high-dimensional data, such as images. The authors plan to investigate how segmentation granularity and occlusion strategies affect user trust, while further refining "no-code" interfaces to make algorithmic transparency accessible to non-programmers.
Summary updated