Beyond Single-Feature Importance with ICECREAM

Michael Oesterle, Patrick Blöbaum, Atalanti-Anastasia Mastakouri, Elke KirschbaumarXiv 2023 · OpenReview

Abstract

Which set of features was responsible for a certain output of a machine learning model? Which components caused the failure of a cloud computing application? These are just two examples of questions we are addressing in this work by Identifying Coalition-based Explanations for Common and Rare Events in Any Model (ICECREAM). Specifically, we propose an information-theoretic quantitative measure for the influence of a coalition of variables on the distribution of a target variable. This allows us to identify which set of factors is essential to obtain a certain outcome, as opposed to well-established explainability and causal contribution analysis methods which can assign contributions only to individual factors and rank them by their importance. In experiments with synthetic and real-world data, we show that ICECREAM outperforms state-of-the-art methods for explainability and root cause analysis, and achieves impressive accuracy in both tasks.

The Problem: The Additivity Trap

Most current Explainable AI (XAI) tools, such as SHAP or LIME, operate on the principle of additivity. They assign an importance score to each individual feature, implying that the total explanation is simply the sum of its parts. However, real-world systems—especially those with redundancies—don't work that way. Imagine a cloud service that only crashes if both Server A and Server B fail. An additive model might tell you both servers are "somewhat" important, but it won't explicitly tell you that the combination of the two is the root cause. ICECREAM addresses this gap by identifying "coalitions" of features whose interplay, rather than individual behavior, drives a specific result.

Methodology: Information-Theoretic Explanations

ICECREAM introduces an information-theoretic explanation score ($E_v$). This score quantifies how much closer the distribution of a target variable ($Y$) gets to the observed outcome ($y$) when you "fix" (intervene on) a specific set of variables ($V_C$).

Key technical components include:

  • Causal Interventions: Unlike methods that simply look at correlations, ICECREAM uses Pearl’s $do$-operator to simulate physical interventions on variables, helping distinguish causes from symptoms.
  • The Explanation Score: Defined using KL divergence (cross-entropy), the score measures the distance between the target's distribution under no intervention versus its distribution when a coalition is fixed. A score of 1.0 means the coalition fully explains the outcome.
Geometric interpretation of the explanation score using KL divergence.
  • Minimal Coalition Search: The algorithm iteratively searches through groups of increasing size to find the smallest set of variables that meets an explanation threshold, adhering to the principle of Occam’s Razor.

Key Insights and Results

Rare Events and Stability

A core advantage of ICECREAM is its handling of rare events. The method satisfies a "desideratum" where rare events leading to an outcome are assigned higher importance than common ones. Furthermore, the authors use stability as a metric for explanation quality: if you fix the features in the explanation and randomize everything else, the model's prediction should remain stable.

Stability of predictions on the South German Credit dataset when randomizing features outside the identified coalition vs. SHAP-ranked features.

Performance Benchmarks

  • Root Cause Analysis (RCA): In cloud computing simulations with 3 or more simultaneous failures, ICECREAM maintained ~100% accuracy in identifying the root cause. In contrast, state-of-the-art RCA methods saw their accuracy plummet to below 40% as complexity increased.
  • Model Stability: On the South German Credit (SGC) dataset, fixing the 3-4 features identified by ICECREAM resulted in significantly more stable predictions than fixing the top features ranked by SHAP.
  • Conciseness: ICECREAM successfully ignored "distractor" features in synthetic benchmarks (CorrAL dataset) that often trip up other selection algorithms.

Limitations and Future Work

While ICECREAM provides a robust framework for coalition-based XAI, it currently relies on discrete target variables and assumes some knowledge of the system's causal structure (a Directed Acyclic Graph). Additionally, searching through all possible coalitions is a combinatorial problem that may become intractable for extremely high-dimensional data, such as high-resolution images, without further search heuristics. Future research will focus on extending the mathematical formulation to continuous regression tasks and improving computational efficiency.

Summary updated

Abstract

Which set of features was responsible for a certain output of a machine learning model? Which components caused the failure of a cloud computing application? These are just two examples of questions we are addressing in this work by Identifying Coalition-based Explanations for Common and Rare Events in Any Model (ICECREAM). Specifically, we propose an information-theoretic quantitative measure for the influence of a coalition of variables on the distribution of a target variable. This allows us to identify which set of factors is essential to obtain a certain outcome, as opposed to well-established explainability and causal contribution analysis methods which can assign contributions only to individual factors and rank them by their importance. In experiments with synthetic and real-world data, we show that ICECREAM outperforms state-of-the-art methods for explainability and root cause analysis, and achieves impressive accuracy in both tasks.
BibTeX
@misc{oesterle2023beyond,
  author = {Michael Oesterle and Patrick Blöbaum and Atalanti-Anastasia Mastakouri and Elke Kirschbaum},
  title = {{Beyond Single-Feature Importance with ICECREAM}},
  year = {2023},
  eprint = {2307.09779},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2307.09779},
}
Figure