SIRUS: making random forests interpretable

Clément Bénard, Gérard Biau, Sébastien Da Veiga, Erwan ScornetarXiv 2019 · OpenReview

Abstract

State-of-the-art learning algorithms, such as random forests or neural networks, are often qualified as "black-boxes" because of the high number and complexity of operations involved in their prediction mechanism. This lack of interpretability is a strong limitation for applications involving critical decisions, typically the analysis of production processes in the manufacturing industry. In such critical contexts, models have to be interpretable, i.e., simple, stable, and predictive. To address this issue, we design SIRUS (Stable and Interpretable RUle Set), a new classification algorithm based on random forests, which takes the form of a short list of rules. While simple models are usually unstable with respect to data perturbation, SIRUS achieves a remarkable stability improvement over cutting-edge methods. Furthermore, SIRUS inherits a predictive accuracy close to random forests, combined with the simplicity of decision trees. These properties are assessed both from a theoretical and empirical point of view, through extensive numerical experiments based on our R/C++ software implementation sirus available from CRAN.

Introduction

In critical domains such as healthcare, criminal justice, and industrial manufacturing, predictive accuracy alone is insufficient. Models must also be interpretable. While state-of-the-art algorithms like random forests and neural networks offer high accuracy, they function as "black boxes," often requiring thousands of operations for a single prediction.

True interpretability necessitates a "triptych" of properties: simplicity (compact model size), predictivity (high accuracy), and stability. Stability is frequently overlooked: if a model's structure changes significantly with minor data perturbations, domain experts may lose trust in its conclusions. Traditional interpretable models, such as decision trees, are notoriously unstable. SIRUS (Stable and Interpretable RUle Set) was developed to bridge this gap, providing a model that is simultaneously simple, accurate, and structurally robust.

Methodology

SIRUS builds upon the core principles of random forests but introduces several key modifications to ensure the resulting rule set is stable. The algorithm breaks down the complexity of random forests into a simple, stable list of human-readable rules.

1. Quantile Discretization

Unlike standard trees that search for the optimal split point among all data values, SIRUS restricts splits to empirical $q$-quantiles (e.g., deciles). This discretization prevents the model structure from "jittering" due to minor data variations, serving as a foundational element for stability.

2. Rule Generation & Selection

The algorithm grows a large forest of shallow trees, typically of depth 2. Each node in these trees represents a "path" or a rule. Instead of complex optimization, SIRUS counts the frequency of each rule across the entire forest.

Partitioning of the input space and the corresponding tree structure used for rule generation.

3. Frequency Thresholding and Aggregation

Rules appearing with a frequency higher than a threshold $p_0$ are selected, as they represent the most robust patterns in the data. Redundant rules (linear combinations of more frequent ones) are removed in a post-treatment phase. The final prediction is a simple average of the outputs of these selected rules, ensuring a transparent decision process.

Experimental Evaluation

The authors evaluated SIRUS using the Titanic dataset, the SECOM industrial dataset (590 variables), and 19 UCI benchmark datasets.

Predictive Performance and Simplicity

On the SECOM dataset, SIRUS achieved a 1-AUC of 0.30, nearly matching the 0.29 of a full Random Forest, while a single CART tree performed poorly (0.48). The algorithm typically produces very short rule sets, often between 6 and 25 rules. For the Titanic dataset, it generated just 7 rules to explain passenger survival.

The 7-rule set generated by SIRUS for the Titanic dataset.

Stability

In cross-validation tests, SIRUS showed significantly higher stability than competitors like RuleFit or Node Harvest. While other methods might return entirely different rules when the data is slightly changed, the core of the SIRUS rule set remains consistent.

Discussion and Future Work

The research demonstrates that stability is a prerequisite for trust; an unstable model provides an "arbitrary analysis" of underlying phenomena. SIRUS is mathematically proven to be asymptotically stable. Furthermore, the study finds that shallow trees (depth 1 or 2) are usually sufficient, as deeper rules are often too sensitive to noise.

While highly effective for classification, future work aims to extend SIRUS to regression tasks. Additionally, for extremely high-dimensional settings, incorporating specific variable selection strategies could further refine performance. Although training costs are higher than single trees due to the forest-based approach, the resulting interpretability and stability provide significant value for critical decision-making applications.

Summary updated

Abstract

State-of-the-art learning algorithms, such as random forests or neural networks, are often qualified as "black-boxes" because of the high number and complexity of operations involved in their prediction mechanism. This lack of interpretability is a strong limitation for applications involving critical decisions, typically the analysis of production processes in the manufacturing industry. In such critical contexts, models have to be interpretable, i.e., simple, stable, and predictive. To address this issue, we design SIRUS (Stable and Interpretable RUle Set), a new classification algorithm based on random forests, which takes the form of a short list of rules. While simple models are usually unstable with respect to data perturbation, SIRUS achieves a remarkable stability improvement over cutting-edge methods. Furthermore, SIRUS inherits a predictive accuracy close to random forests, combined with the simplicity of decision trees. These properties are assessed both from a theoretical and empirical point of view, through extensive numerical experiments based on our R/C++ software implementation sirus available from CRAN.
BibTeX
@misc{benard2019sirus,
  author = {Clément Bénard and Gérard Biau and Sébastien Da Veiga and Erwan Scornet},
  title = {{SIRUS: making random forests interpretable}},
  year = {2019},
  eprint = {1908.06852},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1908.06852},
}
Figure