DALEX: moDel Agnostic Language for Exploration and eXplanation

Przemysław Biecek, Szymon Maksymiuk, Hubert Baniecki, Przemyslaw BiecekOpenalex Unknown 2018 · DOI · OpenAlex

Abstract

Any unverified black box model is the path to failure. Opaqueness leads to distrust. Distrust leads to ignoration. Ignoration leads to rejection. DALEX package xrays any model and helps to explore and explain its behaviour. Machine Learning (ML) models are widely used and have various applications in classification or regression. Models created with boosting, bagging, stacking or similar techniques are often used due to their high performance. But such black-box models usually lack direct interpretability. DALEX package contains various methods that help to understand the link between input variables and model output. Implemented methods help to explore the model on the level of a single instance as well as a level of the whole dataset. All model explainers are model agnostic and can be compared across different models. DALEX package is the cornerstone for 'DrWhy.AI' universe of packages for visual model exploration.

Introduction: The Black Box Dilemma

In contemporary machine learning, a significant tension exists between predictive performance and model interpretability. While complex algorithms like Gradient Boosting, Random Forests, and Deep Neural Networks often yield superior accuracy, they frequently operate as "black boxes." Their internal decision-making logic is opaque, which can lead to a lack of trust among stakeholders and, ultimately, the rejection of the model in critical applications.

The DALEX (moDel Agnostic Language for Exploration and eXplanation) framework addresses this challenge by providing a unified set of tools to "x-ray" any machine learning model. By offering a model-agnostic approach, DALEX allows practitioners to apply consistent diagnostic techniques across diverse model architectures, facilitating direct comparisons and deeper insights into feature-output relationships.

Methodology: Standardizing Model Explanation

The core of the DALEX workflow is the creation of a standardized "explainer" object. This is achieved through the explain() function, which encapsulates the model, the training data, and the prediction logic into a single, manageable entity.

Core parameters of the DALEX explainer function including model, data, and target variable.

As shown in the parameter definitions, the function requires the model object, the dataset (ideally without the target column), and the target vector y. A critical component is the predict_function, which defaults to yhat but can be customized to handle various output formats from different R packages.

Additional configuration parameters for the explainer, such as weights, labels, and verbosity settings.

DALEX provides two primary levels of explanation:

  1. Global Explanations (Dataset Level): These tools assess the model's behavior across the entire dataset. Key methods include permutation-based Variable Importance (model_parts) and Variable Profiles like Partial Dependence Plots (PDP) or Accumulated Local Effects (ALE) via model_profile.
  2. Local Explanations (Instance Level): These focus on individual predictions. Attributions using Break Down plots or SHAP values (predict_parts) decompose a specific prediction into feature contributions, while Ceteris Paribus Profiles (predict_profile) show how a prediction changes when a single variable is varied.

Datasets and Benchmarking

To demonstrate its versatility, DALEX includes several built-in datasets used for demonstration and benchmarking:

  • apartments: An artificial regression dataset designed to mimic complex model responses.
  • dragons: A dataset featuring non-linearity and concept drift.
  • fifa: Real-world player data from FIFA 20.
  • titanic: The classic classification dataset for survival prediction.
  • covid: Simulated mortality data aligned with NIH findings.

Key Insights and Results

DALEX serves as the foundation for the "DrWhy.AI" ecosystem, a collection of R packages dedicated to visual model exploration. Its primary strength lies in its broad compatibility; it seamlessly integrates with models from caret, mlr, xgboost, h2o, and more. By standardizing the "y-hat" (prediction) output, DALEX enables side-by-side comparisons of fundamentally different models using the same visual grammar.

Furthermore, the package introduces "Natural Language Descriptions," which translate complex visual diagnostics into human-readable summaries, bridging the gap between technical data science and non-technical decision-making.

Limitations and Future Directions

While powerful, DALEX's reliance on permutation-based methods and SHAP values can be computationally intensive for very large datasets or complex models. Users are encouraged to utilize sampling parameters (N and n_sample) to mitigate this. Future development continues to focus on deeper integration within the DrWhy.AI universe to provide even more automated and interactive diagnostic tools.

For more details, see Biecek (2018) at https://jmlr.org/papers/v19/18-416.html.

Summary updated

Abstract

Any unverified black box model is the path to failure. Opaqueness leads to distrust. Distrust leads to ignoration. Ignoration leads to rejection. DALEX package xrays any model and helps to explore and explain its behaviour. Machine Learning (ML) models are widely used and have various applications in classification or regression. Models created with boosting, bagging, stacking or similar techniques are often used due to their high performance. But such black-box models usually lack direct interpretability. DALEX package contains various methods that help to understand the link between input variables and model output. Implemented methods help to explore the model on the level of a single instance as well as a level of the whole dataset. All model explainers are model agnostic and can be compared across different models. DALEX package is the cornerstone for 'DrWhy.AI' universe of packages for visual model exploration. Find more details in (Biecek 2018) &lt;<a href="https://jmlr.org/papers/v19/18-416.html" target="_top">https://jmlr.org/papers/v19/18-416.html</a>&gt;.
BibTeX
@inproceedings{biecek2018dalex,
  author = {Przemysław Biecek and Szymon Maksymiuk and Hubert Baniecki and Przemyslaw Biecek},
  title = {{DALEX: moDel Agnostic Language for Exploration and eXplanation}},
  booktitle = {Openalex Unknown},
  year = {2018},
  doi = {10.32614/cran.package.dalex},
}
Figure