Inherently Interpretable Tree Ensemble Learning
Lists
Only you see this
Abstract
Tree ensembles such as random forests and gradient boosting machines are among the most effective methods for tabular prediction, but their strong performance often comes at the cost of interpretability. We show that ensembles of shallow decision trees admit an equivalent functional ANOVA representation, making them inherently interpretable while retaining competitive accuracy. Building on this insight, we develop an exact algorithm that decomposes tree ensembles into main effects and interactions, yielding faithful explanations without approximation. We further introduce two strategies to enhance interpretability: (i) imposing constraints on depth, monotonicity, and interactions, and (ii) post-hoc pruning of trivial effects via sparse modeling and effect selection. Across synthetic and real-world datasets, our approach achieves a superior trade-off between interpretability and predictive power compared to established interpretable models such as Explainable Boosting Machines and GAMI-Net. These results position shallow tree ensembles as a practical and theoretically grounded alternative for transparent high-performance modeling of tabular data.
BibTeX
@misc{yang2026inherently,
author = {Zebin YANG and Agus Sudjianto and Xiaoming Li and Freddy Lecue and Aijun Zhang},
title = {{Inherently Interpretable Tree Ensemble Learning}},
year = {2026},
url = {https://openreview.net/forum?id=zxPXPWdel1},
howpublished = {OpenReview},
note = {Submitted to ICLR 2026},
}