DALEXtra: Extension for 'DALEX' Package

Szymon Maksymiuk, Przemysław Biecek, Hubert Baniecki, Przemyslaw BiecekOpenalex Unknown 2019 · DOI · OpenAlex

Abstract

Provides wrapper of various machine learning models. In applied machine learning, there is a strong belief that we need to strike a balance between interpretability and accuracy. However, in the field of interpretable machine learning, there are more and more new ideas for explaining black-box models implemented in 'R'. 'DALEXtra' creates 'DALEX' (Biecek, 2018) explainers for many types of models, including those created using Python libraries (scikit-learn, Keras) and the Java-based H2O library. A key feature of the package is the Champion-Challenger analysis and an innovative approach to model performance across subsets of test data via Funnel Plots.

Introduction

The modern machine learning landscape is characterized by a persistent tension between predictive accuracy and model interpretability. While complex "black-box" models—such as deep neural networks or gradient-boosted machines—often yield the highest performance, their internal logic remains opaque. Furthermore, the field is technically fragmented; data scientists frequently utilize disparate ecosystems, training models in Python (scikit-learn, Keras), Java (H2O), or R (tidymodels, mlr3). This fragmentation complicates the standardized auditing and comparison of models.

DALEXtra addresses these challenges by serving as a universal bridge between these diverse environments and the DALEX (Descriptive mAchine Learning EXplanations) framework. By providing a unified interface, it allows for consistent interpretability analysis regardless of the model's origin.

Methodology: Unified Model Wrapping

The primary contribution of DALEXtra is its suite of specialized "explainer" wrappers. These functions abstract the complexities of cross-language communication, enabling R users to audit models from other platforms seamlessly.

For instance, the package leverages the reticulate library to interface with Python-based models via explain_scikitlearn and explain_keras. Similarly, it provides robust support for H2O models through the explain_h2o function.

The explain_h2o function header in DALEXtra.

The design of these wrappers ensures that metadata, such as model type (classification or regression) and diagnostic verbosity, is handled consistently across different backends. This standardization is critical for ensuring that subsequent explanations (like SHAP or partial dependence plots) are comparable across models.

Parameters for the explain_h2o wrapper.

Model Comparison and Auditing

Beyond simple wrapping, DALEXtra introduces advanced workflows for model selection and validation:

  1. Champion-Challenger Analysis: This framework facilitates a formal competition between a "Champion" (the current production model) and "Challengers" (newly developed candidates). The package automates the generation of comprehensive reports that compare these models across multiple performance and reliability metrics.
  2. Funnel Plots: A significant innovation in DALEXtra is the funnel_measure function. Unlike global metrics (e.g., overall RMSE), Funnel Plots partition test data into quantiles based on specific feature variables. This allows researchers to identify if a model's superiority is universal or if it fails in specific data sub-regions, such as high-value transactions or specific demographic groups.

Discussion and Limitations

While DALEXtra significantly simplifies the model auditing process, it is not without challenges. Dependency management remains a hurdle, particularly when orchestrating Python environments from within R. Although DALEXtra provides utility functions like create_env to mitigate this, users must maintain environment consistency. Additionally, the computational cost of generating instance-level explanations (like LIME) can be substantial for large-scale datasets.

Future developments in the ModelOriented ecosystem are expected to further integrate these tools with automated fairness and bias detection, continuing the mission to make complex machine learning models more transparent and accountable.

Summary updated

Abstract

Provides wrapper of various machine learning models. In applied machine learning, there is a strong belief that we need to strike a balance between interpretability and accuracy. However, in field of the interpretable machine learning, there are more and more new ideas for explaining black-box models, that are implemented in 'R'. 'DALEXtra' creates 'DALEX' Biecek (2018) &lt;<a href="https://doi.org/10.48550/arXiv.1806.08915" target="_top">doi:10.48550/arXiv.1806.08915</a>&gt; explainer for many type of models including those created using 'python' 'scikit-learn' and 'keras' libraries, and 'java' 'h2o' library. Important part of the package is Champion-Challenger analysis and innovative approach to model performance across subsets of test data presented in Funnel Plot.
BibTeX
@inproceedings{maksymiuk2019dalextra,
  author = {Szymon Maksymiuk and Przemysław Biecek and Hubert Baniecki and Przemyslaw Biecek},
  title = {{DALEXtra: Extension for 'DALEX' Package}},
  booktitle = {Openalex Unknown},
  year = {2019},
  doi = {10.32614/cran.package.dalextra},
}
Figure