Functional ANOVA Decomposition for Model Interpretability

This direction is split into other directions: functional ANOVA decomposition. Its last version stays readable and citable here.

Version 1 of 157 papers66 conceptsfrozen 2026-10-04

In machine learning, we often face a trade-off between the simplicity of a linear model and the predictive power of a "black box" like a deep neural network or a gradient-boosted tree. While black boxes capture complex, non-linear relationships, they offer little insight into how they arrive at a decision.

Functional ANOVA (Analysis of Variance) decomposition provides a mathematical bridge between these two worlds. It allows researchers to take a complex, high-dimensional function and break it down into a sum of simpler, interpretable components: individual "main effects" and multi-way "interactions."

The Core Intuition: Decomposing Complexity

The fundamental idea, as explored by Grace Wahba and colleagues in Structured Machine Learning for Soft Classification with Smoothing Spline ANOVA, is to represent a multivariate function $f(x)$ as a structured sum:

$$f(x) = f_0 + \sum f_i(x_i) + \sum f_{ij}(x_i, x_j) + \dots$$

In this framework:

  • $f_0$ is the global intercept (the average prediction).
  • $f_i(x_i)$ are main effects, showing how a single feature influences the outcome in isolation.
  • $f_{ij}(x_i, x_j)$ are pairwise interactions, capturing the "super-additive" effect when two features work together.

By isolating these terms, we can visualize a model's logic through 1D and 2D plots. As Christoph Molnar notes in Quantifying Model Complexity via Functional Decomposition, this allows us to quantify exactly how much of a model's "wiggliness" or complexity comes from simple relationships versus tangled interactions.

Ensuring Identifiability through Purification

A major challenge in functional decomposition is ensuring that the components are "pure." If a main effect $f_i$ and an interaction $f_{ij}$ both capture the same information, the explanation becomes redundant and misleading.

To solve this, researchers like Ming Yuan in Nonnegative Garrote Component Selection in Functional ANOVA models and Linwei Hu in Interpretable Machine Learning based on Functional ANOVA Framework utilize hierarchical purification. This technique projects interaction terms onto a space orthogonal to the main effects. It ensures that an interaction term only represents variation that cannot be explained by the individual variables alone. This mathematical "cleaning" process is what makes the resulting visualizations honest representations of the model's internal logic.

Scaling to High Dimensions

Historically, Smoothing Spline ANOVA (SS-ANOVA) was computationally expensive, scaling poorly with large datasets. Recent work has focused on making these decompositions efficient enough for modern "big data" contexts.

  • Linear Time Estimation: Jinyang Liu, Marvin N. Wright, and Munir Hiabu introduced FastPD in Fast Estimation of Partial Dependence Functions using Trees. This algorithm achieves linear time complexity for estimating partial dependence, allowing for rapid functional decomposition even in tree-based ensembles like XGBoost.
  • Kernel Tricks: In The SKIM-FA Kernel, Raj Agrawal and Tamara Broderick developed a method to discover non-linear interactions in linear time relative to the number of features, bypassing the combinatorial explosion that usually occurs when looking for feature pairs.

Beyond Regression: A Unified Lens for XAI

Functional ANOVA is increasingly used as a "unifying map" for the entire field of Explainable AI (XAI). In Unifying Feature-Based Explanations with Functional ANOVA and Cooperative Game Theory, Fabian Fumagalli and colleagues show that many popular methods—like SHAP, PDP, and SAGE—are actually different ways of looking at the same functional decomposition.

This perspective has allowed researchers to extend interpretability into specialized domains:

By treating a model not as a single opaque block, but as a sum of identifiable parts, Functional ANOVA transforms "black-box" machine learning into a transparent, modular system. Readers can see not just that a model works, but the specific hierarchy of features and interactions that make it work.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_functional_anova_decomposition_for_model_fcf45d20,
  title = {Functional ANOVA Decomposition for Model Interpretability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/functional-anova-decomposition-for-model-interpretability/rdr_fcf45d20ac3845a5a27de3bc81d2442c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_functional_anova_decomposition_for_model_fcf45d20_v1,
  title = {Functional ANOVA Decomposition for Model Interpretability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/functional-anova-decomposition-for-model-interpretability/rdr_fcf45d20ac3845a5a27de3bc81d2442c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}