TL;DR
Researchers conducted a massive benchmarking study comparing 12 different methods for making Random Forests (RFs) interpretable. They found that no single method wins across the board; instead, the "best" choice depends on whether you care more about mimicking the RF's predictions, understanding its internal logic, or keeping the explanation as simple as possible.
What Problem Does This Paper Address?
Random Forests are the "workhorses" of tabular data science because they handle non-linear relationships and complex interactions with ease. However, they are fundamentally "black boxes"—an ensemble of hundreds of trees is nearly impossible for a human to audit.
To fix this, researchers use interpretable approximations: simpler models (like a single decision tree or a set of rules) that try to summarize what the complex forest is doing. While many such methods exist, practitioners are often left guessing which one to use. This paper provides the first comprehensive, large-scale empirical comparison to guide that choice, moving beyond small-scale studies to look at prediction similarity, structural similarity, and model complexity simultaneously.
Methods
The authors compared 12 interpretation methods, categorized into three main approaches:
- Oracle Coaching (Surrogate Models): These methods treat the RF as an "oracle." They take the RF’s predictions and train a simpler model (like a Linear Model, SLIM, GUIDE, or CTree) to mimic those outputs.
- Representative Trees: These methods look "under the hood" of the RF. MRT (Most Representative Tree) selects one actual tree from the forest that best represents the whole, while ART (Artificial Representative Tree) constructs a new tree designed to be structurally similar to the entire ensemble.
- Rule Extraction: These methods, such as inTrees, SIRUS, and Node Harvest (NH), extract a small, human-readable set of "if-then" rules directly from the forest's logic.
The study used a standard CART decision tree as a baseline and evaluated everything based on five dimensions: prediction similarity to the RF, structural similarity (did it use the same variables?), model complexity (how many rules/splits?), prediction performance on new data, and runtime.
Datasets
The evaluation was exceptionally rigorous, utilizing:
- 11 Simulation Settings: These were designed with known "ground truths" (e.g., XOR problems, linear effects, and interaction effects) to see if the methods could actually recover the true underlying signals and ignore noise.
- 67 Benchmark Datasets: A collection of 28 regression and 39 classification tasks pulled from the UCI Machine Learning Repository and OpenML, covering diverse fields from medical biometry to social sciences.
Key Insights and Ideas
- The "What" vs. The "How": A major "aha moment" in the paper is that prediction similarity does not equal structural similarity. A surrogate model might perfectly mimic an RF's output while using completely different variables. If your goal is to understand why a model made a decision, you cannot rely on prediction accuracy alone.
- The Compactness King: If you need the shortest possible explanation, ART is the clear winner. It consistently produced the most "compact" models (fewest splits/rules) across almost all scenarios.
- Regression vs. Classification Split: The best tool for the job changes with the outcome type. SLIM (Surrogate Locally-Interpretable Models) dominated for regression similarity, while inTrees was the standout for classification.
- Robustness Matters: Several sophisticated methods (like Node Harvest) frequently failed to produce results on certain datasets due to numerical errors or runtime limits. Sometimes, a simpler "Oracle Coaching" approach using a standard decision tree (OC-CART) is the most reliable choice.
Key Results
- For Regression: SLIM provided the best balance of mimicking the RF and performing well on test data. However, if the data is mostly linear, a simple Linear Model (LM) with forward selection is often just as good and much easier to explain.
- For Classification: inTrees achieved the strongest overall performance on benchmarks. CTree and OC-CART were identified as robust, faster alternatives for larger datasets.
- Structural Fidelity: MRT was the best at reproducing the actual variable usage of the RF in classification simulations, making it the preferred choice for "feature importance" style explanations.
- Efficiency: The fastest methods were the simplest ones—OC-CART and the reference decision tree—while RuleFit and ART were significantly slower, sometimes exceeding the 24-hour runtime limit on complex datasets.
Limitations and Future Work
The authors noted that their study did not involve extensive hyperparameter tuning for the interpretation methods, instead relying on recommended defaults to reflect "typical" use. Future work could explore if tuning significantly changes the rankings.
Additionally, the study highlighted a gap in complexity measures: it is currently difficult to fairly compare the "interpretability" of a linear model (coefficients) against a decision tree (splits) or a rule set (conditions). Developing a unified metric for "human-readability" remains an open challenge for the field. Finally, the authors suggest that future selection procedures should be "data-driven," perhaps using cross-validation to pick the best interpretation method for a specific new dataset.
Summary updated