arenar: Arena for the Exploration and Comparison of any ML Models

Piotr Piątyszek, Przemysław Biecek, Przemyslaw BiecekOpenalex Unknown 2020 · DOI · OpenAlex

Abstract

Generates data for challenging machine learning models in 'Arena' https://arena.drwhy.ai - an interactive web application. You can start the server with XAI (Explainable Artificial Intelligence) plots to be generated on-demand or precalculate and auto-upload data file beside shareable 'Arena' URL.

Introduction

As machine learning models grow in complexity, they increasingly function as "black boxes," obscuring the logic behind specific predictions and complicating model comparison. While various Explainable AI (XAI) libraries exist, they often require significant boilerplate code to generate individual visualizations. arenar is an R package designed to address this by providing a unified, interactive interface for side-by-side model comparison and deep-dive exploration. By leveraging the DrWhy.ai ecosystem, it allows researchers to "push" multiple models and datasets into a single environment where they can be analyzed dynamically without additional plotting scripts.

Methods and Workflow

The arenar package acts as an orchestrator for the DALEX (Descriptive mAchine Learning EXplanations) ecosystem. Its architecture follows a structured "push" workflow:

  1. Initialization: Users define an arena object using create_arena(). This step establishes whether the output will be a live server (using a plumber API) or a static JSON file for sharing.
  2. Data Integration: Models, wrapped as DALEX explainers, and specific observations are integrated into the arena object via functions like push_model() and push_observations().
  3. XAI Generation: The package utilizes specialized libraries—such as ingredients for global profiles and iBreakDown for local explanations—to calculate SHAP values, Break Down plots, Ceteris Paribus profiles, Feature Importance, and Partial Dependence Plots (PDP).
  4. Deployment: Results are either served locally for real-time interaction or exported to the Arena web application.
Core parameters for initializing an Arena object, including explainer definitions and data binning settings.

Key Capabilities and Insights

The "Arena" Concept

Unlike traditional tools that view models in isolation, arenar treats models as competitors. This facilitates direct comparison of how different architectures (e.g., Random Forest vs. Gradient Boosting Machines) handle the same specific observations. Users can modify observation values in real-time through the web interface to conduct "What-If" analysis, observing how model predictions shift dynamically.

Advanced XAI and Fairness

The package provides a comprehensive suite of XAI techniques. By integrating fairmodels, arenar also makes algorithmic bias checks a standard part of the comparison process. Users can configure specific parameters for these calculations, such as the number of permutations for feature importance or the number of random paths for SHAP values.

Advanced configuration parameters for XAI generation, including SHAP paths, feature importance rounds, and fairness cutoffs.

Dual-Mode Workflow

The choice between live and static modes offers significant flexibility. The live mode is optimized for local development and deep data exploration, while the static mode enables the pre-calculation of explanations, allowing them to be shared with stakeholders who do not have R installed.

Limitations and Future Work

While powerful, arenar has several constraints. Parallel computations (using the cl argument) are currently unsupported in the live server mode. Additionally, the run_server function requires the use of IP addresses rather than hostnames. Finally, the static mode requires pre-calculating all plots; for very large datasets or numerous models, this can lead to substantial JSON file sizes, potentially impacting the ease of sharing and web performance.

Summary updated

Abstract

Generates data for challenging machine learning models in 'Arena' &lt;<a href="https://arena.drwhy.ai" target="_top">https://arena.drwhy.ai</a>&gt; - an interactive web application. You can start the server with XAI (Explainable Artificial Intelligence) plots to be generated on-demand or precalculate and auto-upload data file beside shareable 'Arena' URL.
BibTeX
@inproceedings{piatyszek2020arenar,
  author = {Piotr Piątyszek and Przemysław Biecek and Przemyslaw Biecek},
  title = {{arenar: Arena for the Exploration and Comparison of any ML Models}},
  booktitle = {Openalex Unknown},
  year = {2020},
  doi = {10.32614/cran.package.arenar},
}
Figure