Deep neural networks are famously powerful but notoriously opaque. For years, if you wanted to understand why a model predicted a specific outcome—a field known as Explainable AI (XAI)—you likely had to leave the R ecosystem and head over to Python. While bridges like reticulate exist, they often come with a "Python tax" of complex dependencies and environment management.
In their paper, "Interpreting Deep Neural Networks with the Package innsight," Niklas Koenen and Marvin N. Wright introduce a game-changer for the R community: a native, high-performance, and library-agnostic toolbox for deep learning interpretability.
TL;DR
The innsight package is the first unified R framework for feature attribution in neural networks. It allows researchers to peek inside "black box" models from keras, torch, or neuralnet using a fast C++ backend (LibTorch) to provide clear, interactive visualizations of which input variables drive a model's predictions.
What Problem Does This Paper Address?
As neural networks move into high-stakes domains like healthcare and autonomous systems, "because the model said so" is no longer an acceptable answer. We need feature attribution methods—techniques that assign a "relevance score" to each input (like a specific pixel in an image or a variable in a spreadsheet) to show its contribution to the final prediction.
Until now, the most popular tools for this (like captum or innvestigate) were Python-exclusive. R users were stuck between a rock and a hard place: either use simpler, slower "model-agnostic" methods that don't look at the network's internal structure, or deal with the headache of cross-language integration. innsight fills this gap by providing a native R solution that is both deep-learning-library-agnostic and computationally efficient.
Methods
The authors implement a "3-step workflow" that standardizes the XAI process:
- Convert: A
convert()function takes a trained model (from Keras, Torch, etc.) and replicates it into an internaltorch(R version) structure. - Apply: Users choose from a suite of state-of-the-art attribution methods.
- Visualize: The package generates high-quality
ggplot2or interactiveplotlycharts.
Key algorithms included: * Gradient-based: Vanilla Gradients, SmoothGrad, Integrated Gradient, and Expected Gradient. * Rule-based: Layer-wise Relevance Propagation (LRP) with various rules (simple, $\epsilon$, $\alpha$-$\beta$) and DeepLift. * Classic/Agnostic: Connection Weights (CW), plus wrappers for LIME and KernelSHAP.
What makes innsight unique is its use of the R torch package. By building on LibTorch (the C++ backend of PyTorch), it performs heavy-duty array calculations without ever needing a Python installation on the user's machine.
Datasets
The authors validated innsight using three distinct scenarios:
- Bike Sharing Dataset: A regression task used to demonstrate tabular data interpretation.
- Palmer Penguins: A classification task used to show how the model distinguishes between species based on physical traits.
- SIIM-ISIC Melanoma Dataset: A complex, multimodal dataset (images of skin lesions + tabular patient data). This was used to showcase the package's ability to handle high-dimensional image data and combined inputs.
Key Insights and Ideas
- The "Universal Translator" for Models: The most clever part of
innsightis its ability to ingest models from different libraries. By converting them into a unified internal format, it ensures that the same XAI method can be applied to a Keras model and aneuralnetmodel with identical syntax. - Simultaneous Multi-Output Calculation: Unlike many Python counterparts that calculate attributions for one output class at a time,
innsightis optimized to calculate relevances for multiple output nodes simultaneously, significantly speeding up the process for multi-class problems. - Interactive XAI: By integrating with
plotly, the package allows users to hover over pixels or bars to see exact relevance values, making the "explanation" much more exploratory and user-friendly.
Key Results
- Numerical Precision: The authors compared
innsightagainst five major Python libraries. The results were nearly identical, with mean absolute differences staying below $10^{-6}$ (the threshold for floating-point errors), proving that the R implementation is as reliable as the industry standards. - Performance:
innsightproved to be significantly faster than Keras-based Python tools (innvestigate,deeplift). While slightly slower than the highly optimizedcaptum(PyTorch) for some tasks, it was competitive and sometimes faster when dealing with multiple output nodes. - Multimodal Success: In the melanoma example,
innsightsuccessfully highlighted the "ragged borders" of malignant lesions as high-relevance areas, aligning with how human dermatologists actually diagnose skin cancer.
Limitations and Future Work
While a massive step forward, the authors note a few current boundaries:
- Sequential Only: Currently, the automatic converter primarily supports sequential models. Complex, non-sequential
nn_modulesfrom the Rtorchpackage require manual list-based conversion. - CPU Bound: The current version of
innsightsupports CPU calculations only; GPU support is a target for future updates. - Method Expansion: The authors plan to add more "perturbation-based" methods like Occlusion or RISE in future iterations.
For R users in data science and bioinformatics, innsight represents the end of "Python envy" for model interpretability. It’s a robust, fast, and deeply integrated tool that finally brings the "why" of deep learning to the R console.
Summary updated