TL;DR
The ONAM R package introduces a robust framework for cracking open "black-box" machine learning models using Orthogonal Neural Additive Models. By decomposing complex predictions into unique, identifiable main effects and interactions, it allows researchers to visualize exactly how a model makes decisions without sacrificing predictive power.
What Problem Does This Paper Address?
Modern machine learning—think XGBoost, Random Forests, or Deep Neural Networks—is incredibly accurate but notoriously opaque. In high-stakes fields like medicine or insurance, "it just works" isn't a good enough answer; we need to know why.
Existing Interpretable Machine Learning (IML) tools like Partial Dependence Plots (PDPs) or LIME provide glimpses into model behavior, but they often struggle with functional decomposition—the process of breaking a prediction function into a sum of simpler subfunctions (main effects and interactions). The core challenge is identifiability: without strict mathematical constraints, a decomposition isn't unique. You could potentially "hide" a simple main effect inside a complex interaction term, making the model look more mysterious than it actually is.
Methods
The authors present a three-step post-hoc approach implemented in the ONAM package:
- Explanation Sampling: First, they generate a dataset of predictions from the original black-box model. This makes the method model-agnostic; it doesn't care if the underlying model is a neural net or a boosted tree.
- NAM Fitting: They fit a Neural Additive Model (NAM) to these predictions. In a NAM, each feature (or pair of features) is processed by its own small neural network. The final prediction is simply the sum of these individual outputs.
- Orthogonalization: This is the "secret sauce." The authors apply an iterative algorithm to ensure hierarchical orthogonality. This mathematical constraint forces lower-order effects (like the impact of a single variable) to capture as much information as possible before higher-order interactions are allowed to explain the "leftovers."
Datasets
The paper demonstrates the package using two classic benchmarks:
- Boston Housing Dataset: A regression task predicting median home values based on 13 socioeconomic features.
- Cleveland Heart Disease Dataset: A binary classification task used to diagnose coronary artery disease.
Key Insights and Ideas
- Unique Decompositions: By satisfying hierarchical orthogonality, ONAM guarantees that the resulting interpretation is unique. This removes the ambiguity found in many other surrogate modeling techniques.
- The Optimality Principle: The method adheres to the idea that we should explain as much as possible with the simplest terms first. If a relationship can be explained as a main effect, ONAM won't let it be "absorbed" into a complex interaction.
- Quantifying Interpretability: The authors introduce a "Degree of Interpretability" metric ($I_k$). This tells you exactly what percentage of your black-box model's variance is captured by simple, interpretable components.
- Efficiency: While training neural networks can be slow, the actual orthogonalization step is computationally cheap, typically taking less than 1% of the total model-fitting time.
Key Results
In the Boston Housing example, the authors used ONAM to decompose a high-performing XGBoost model.
- Interpretability Score: They found that 95.8% of the XGBoost model's predictions could be explained by just main effects and two-way interactions.
- Visualization: The package produces clear, actionable plots. For instance, it revealed a clear negative relationship between the "pupil/teacher ratio" and house prices, and a complex interaction between the "number of rooms" and the "age of the house."
For the Heart Disease dataset, ONAM successfully handled binary classification by performing the decomposition on the logit scale, allowing for interpretations in terms of odds ratios.
Limitations and Future Work
- The Curse of Dimensionality: The number of possible interactions grows exponentially with the number of features. For models with hundreds of variables, users must manually specify which interactions are of interest to keep the computation feasible.
- Stability: Neural networks are sensitive to initialization. The authors recommend using an ensemble of NAMs to ensure the resulting interpretations are stable and reproducible.
- Python Dependency: While the package is for R, it relies on a Python backend (via
keras3). Users need a working Python environment, though the package provides helper functions to set this up.
Summary updated