Higher-Order Component Attribution via Kolmogorov–Arnold Networks

Samy Mammeri, Christian GagnéMechInterp @ NeurIPS 2025 · OpenReview

TL;DR

This paper introduces a novel framework for understanding how different parts of a deep learning model (components) interact to produce a prediction. By using Kolmogorov-Arnold Networks (KANs) to model these interactions non-linearly, they achieve more accurate predictions and can uncover complex, higher-order relationships between components, going beyond the limitations of existing linear methods.

What Problem Does This Paper Address?

Deep learning models are becoming increasingly complex, making it difficult to pinpoint which specific components (like transformer layers or residual blocks) are responsible for a given prediction. Current methods for "component attribution" often simplify this by assuming that components contribute independently and linearly to the final output. This is a significant oversimplification, as the real magic often happens in the interactions between these components. These interactions can be complex and nonlinear, and ignoring them means we miss a crucial part of how models make decisions. This paper aims to address this gap by developing a method that can explicitly model and quantify these higher-order, nonlinear interactions.

Methods

The core of this work is the application of Kolmogorov-Arnold Networks (KANs) as a more expressive surrogate model for component attribution. Here's a breakdown of their approach:

  1. Perturbation-Response Dataset Generation: For a given input and a trained model, they systematically perturb the outputs of individual components by adding noise. They then record how these perturbations affect the model's final prediction. This creates a dataset that captures the relationship between component-level changes and the overall model response.
  2. KAN as a Nonlinear Surrogate: Instead of fitting a simple linear model (like previous methods), they train a KAN on this perturbation-response data. KANs are known for their ability to represent complex, nonlinear functions efficiently, making them ideal for capturing the intricate ways components interact.
  3. Extracting Interaction Coefficients: They use two complementary methods to extract these interactions from the trained KAN:
  4. Automatic Differentiation: They directly compute derivatives of the KAN with respect to component perturbations. These derivatives, especially mixed partial derivatives, quantify the influence of multiple components acting together.
  5. Symbolic Regression: They fit a closed-form symbolic representation of the KAN. This allows them to derive analytical formulas for interaction scores, which can be more interpretable and robust.

This approach allows them to quantify not just the independent effect of a component (first-order attribution) but also how pairs of components interact (second-order), triplets, and so on, up to a specified order.

Datasets

The paper evaluates their approach on standard image classification tasks using the following model-dataset pairs:

  • ResNet-18 trained on CIFAR-10
  • ResNet-50 trained on ImageNet
  • ViT-B/16 trained on ImageNet

They generate localized perturbation-response datasets for each of these setups. No new benchmarks are introduced; they adapt existing experimental frameworks.

Key Insights and Ideas

  • Nonlinearity is Crucial: The most significant insight is that linear models are insufficient for capturing component interactions. The paper demonstrates that nonlinear modeling, specifically with KANs, is essential for accurately representing how model components influence predictions.
  • KANs as Powerful Attribution Tools: KANs prove to be highly effective surrogates for modeling complex component interactions. Their ability to learn and represent nonlinear relationships directly translates to more accurate attribution scores.
  • Hierarchical Understanding of Interactions: By computing mixed partial derivatives, the method provides a hierarchical view of component influence, distinguishing between independent effects and complex joint contributions. This offers a richer understanding than traditional methods.
  • Bridging to Formal Verification: The authors highlight the potential for their symbolic interaction scores to be integrated with formal verification methods, opening doors for more rigorous analysis of model behavior.

Key Results

  • Improved Attribution Accuracy: The KAN-based approach consistently outperforms the linear baseline (COAR) in terms of attribution accuracy, as measured by Pearson correlation coefficient and mean squared error between predicted and ground-truth responses. The paper shows significant improvements across all evaluated image classification tasks.
  • Extraction of Higher-Order Interactions: The method successfully enables the extraction of higher-order component interactions, which are missed by linear attribution techniques.
  • Validation on Symbolic Functions: Preliminary experiments on random 3-variable symbolic functions show that KAN surrogates can accurately recover mixed partial derivatives up to third order using both automatic and symbolic differentiation.

Limitations and Future Work

  • Dimensionality Constraints: A key limitation acknowledged is the current inability of KANs to efficiently handle very high-dimensional inputs. This restricts the granularity of components that can be analyzed (e.g., they use entire layers or blocks rather than individual neurons).
  • Scalability: Relaxing the dimensionality constraints of KANs is a primary direction for future work, aiming to extend the method's reach to finer-grained component analysis.
  • Integration with Formal Verification: The authors express interest in future work that integrates their symbolic interaction scores with formal verification methods to support richer counterfactual analyses.

Summary updated

Abstract

Component attribution quantifies how model components, from individual neurons to transformer layers, contribute to a prediction. Despite their successes, most methods assume additive linear effects between components and overlook interactions that shape how predictions arise from internal computations. In this work, we formalize nonlinear component modeling and introduce a Kolmogorov–Arnold Network (KAN)-based framework for component attribution. We fit KAN surrogates on perturbation–response data to represent effects nonlinearly, then use them to extract local component interaction coefficients in two complementary ways: by automatic differentiation of the trained KAN and by recovering a symbolic surrogate whose closed-form mixed partial derivatives yield symbolic interaction scores. This provides a way to relate a classifier's output back to interacting internal building blocks instead of isolated components. The resulting expressions are intended for future integration with formal verification methods to support richer counterfactual analyses. Preliminary results on standard image classification models demonstrate that our approach improves the accuracy of counterfactual predictions and enables extraction of higher-order component interactions compared to linear attribution.
BibTeX
@inproceedings{mammeri2025higher,
  author = {Samy Mammeri and Christian Gagné},
  title = {{Higher-Order Component Attribution via Kolmogorov–Arnold Networks}},
  booktitle = {MechInterp @ NeurIPS},
  year = {2025},
  url = {https://openreview.net/forum?id=xzLOhw7LQm},
}
Figure