Feature Attribution with Shapley Additive Explanations

18 concepts18 papers1 proposal

Modern machine learning models routinely achieve remarkable accuracy across diverse domains, from predicting student dropout rates (Dass et al.) and detecting IoT reconnaissance attacks (Alani) to diagnosing complex medical conditions (Dell’Era et al.; Zhang et al.). However, these models—such as deep neural networks, LightGBM, and ensemble random forests—frequently operate as "black boxes." They hide the underlying logic of their predictions behind thousands or millions of parameters.

To trust and safely deploy these models in high-stakes environments, we must understand not just what a model predicts, but why. Traditional feature importance metrics (such as tree-based split counts or permutation importance) often fail because they evaluate features in isolation or depend heavily on the specific structure of the model, leading to inconsistent and misleading attributions when features interact.

Shapley Additive Explanations (SHAP) solves this problem by framing feature attribution through the lens of cooperative game theory. It provides a mathematically rigorous framework that unifies local explanations with global model interpretability.

The Intuition: Cooperative Games and Marginal Contributions

To understand why SHAP works, imagine a cooperative game where a team of players collaborates to achieve a specific score, and a cash prize is awarded based on that final score. If the players possess different skills and interact with each other in complex ways, how do you fairly distribute the prize among them?

In machine learning, we can map this scenario directly to model predictions:

  • The Players: The individual input features of a specific data instance.
  • The Game: The trained machine learning model.
  • The Prize: The difference between the model's actual prediction for that instance and the expected (average) prediction across the entire dataset.

If we evaluate a feature's importance solely by dropping it from the model, we ignore feature interactions. For example, a medical model might rely on both a patient's age and a specific blood biomarker. Evaluating either feature alone might yield a negligible change in the prediction, yet their combination could be highly predictive.

SHAP resolves this by calculating the marginal contribution of a feature across all possible subsets (coalitions) of features. For a given feature, SHAP evaluates how much the model's prediction changes when that feature is added to a subset $S$ that does not already contain it. By computing this change across every possible combination of the remaining features and taking a weighted average, SHAP determines the feature's true impact on the outcome.

The Mathematical Guarantees of Fairness

SHAP is uniquely powerful because it is the only local feature attribution method that satisfies a set of highly desirable mathematical properties derived from Shapley value estimation:

  1. Efficiency (Local Accuracy): The sum of the SHAP values for all features must equal the difference between the model's local prediction and the expected base prediction. The explanation perfectly accounts for the model's output deviation.
  2. Symmetry: If two features contribute identically to all possible coalitions, their SHAP values must be equal.
  3. Dummy (Null Player): A feature that never changes the model's prediction, regardless of which coalition it joins, receives a SHAP value of zero.
  4. Additivity: If a prediction is the result of summing two separate model outputs, the SHAP values of the combined model are simply the sum of the SHAP values from the individual models.

By adhering to these axioms, SHAP guarantees consistency. If a model changes such that a specific feature's marginal contribution increases or stays the same, its calculated SHAP value is guaranteed not to decrease. Traditional importance metrics routinely violate this property.

From Local Explanations to Global Insights

A defining characteristic of SHAP is its ability to bridge the gap between local and global interpretability. Because SHAP computes an additive attribution score for each individual prediction, these local values can be aggregated across an entire dataset to construct a comprehensive, global view of model behavior.

Interpreting Complex Clinical Models

This duality is highly valuable in clinical diagnostics, where physicians require both a macro-level understanding of a disease's drivers and a patient-specific rationale for a diagnosis.

For instance, Yang et al. utilized SHAP values to interpret a Random Forest model diagnosing heart failure from oral hyperspectral images, ensuring that the specific spectral bands driving individual patient diagnoses aligned with physiological expectations. Similarly, Wang et al. applied SHAP to a LightGBM model predicting post-hepatectomy liver failure, transforming a complex gradient-boosted tree structure into transparent, quantifiable risk factors that clinicians can evaluate at the bedside.

The methodology scales seamlessly across diverse data modalities:

  • Radiomics and Imaging: In oncology and structural imaging, the authors of several recent studies have used SHAP to identify which spatial and temporal features dictate model predictions. Examples include detecting temporomandibular joint disc displacement (Ji et al.), predicting the success of shockwave lithotripsy for ureteral stones (Ye et al.), and evaluating temporal CT changes to predict lung cancer treatment responses (Zheng et al.).
  • Molecular Biomarkers: When dealing with high-dimensional biological data, such as small RNA signatures in glaucoma (Dobrzycka et al.), SHAP isolates the specific molecular expressions that differentiate pathologies.
  • Time-Series ICU Data: For deep learning architectures like recurrent neural networks tracking continuous intensive care unit data, Bremer et al. demonstrated that SHAP can pinpoint the exact clinical windows and variables that trigger early warnings for shunt dependency.

Beyond Interpretation: Feature Selection and Discovery

While SHAP was designed for model interpretability, its rigorous feature attribution properties make it an effective tool for scientific discovery and feature engineering. By identifying exactly which variables drive a model's predictive power, researchers can filter out uninformative noise and uncover physical or biological mechanisms.

Geotechnical and Environmental Engineering

In fields governed by physical laws, SHAP validates whether a machine learning model has learned physically meaningful relationships rather than exploiting spurious correlations. Xu et al. used SHAP to quantify how sand-clay ratios and stress states influence the mechanical strength of marine mixtures, using these insights to derive simplified empirical formulas.

In digital agriculture, Xiang et al. combined hyperspectral data with an optimized XGBoost model, using SHAP to isolate the specific spectral wavelengths most sensitive to soil nutrient variability. Similarly, when deep learning models (such as Conv1d-GRU architectures) are deployed to identify mine water inrush sources, the authors used SHAP to map complex chemical interactions back to identifiable hydrochemical profiles (Li 0030 et al.).

Isolating Toxicological and Biological Drivers

When analyzing environmental health hazards, mixtures of variables often obscure individual effects. Zhang et al. leveraged SHAP to untangle the toxicological effects of metal mixtures in cardiovascular-kidney-metabolic syndrome, successfully isolating cobalt as a stage-specific driver of the disease.

Finally, this attribution capability can be integrated directly into machine learning pipelines to optimize performance. In gene function prediction, Romero 0005 et al. demonstrated that SHAP values can serve as an explicit feature filtering mechanism. By ranking and selecting features based on their mathematical contribution to hierarchical multi-label classifications, the authors systematically reduced computational complexity and noise while maintaining high prediction accuracy.

Application Domain Model Type Role of SHAP Key Reference
Oncology Temporal Radiomics (CT) Preoperative prediction of chemoimmunotherapy response Zheng et al.
Toxicology Ensemble Learning Isolating specific metal toxicity in complex mixtures Zhang et al.
Geotechnics LDA-SVM Hybrid Quantifying drivers of coral sand-clay mixture strength Xu et al.
Bioinformatics Hierarchical Multi-Label Filtering co-expression features for gene function prediction Romero 0005 et al.

By framing feature importance as a cooperative game, SHAP provides a mathematically sound method for model interpretability. It ensures that feature attributions are fair, consistent, and locally accurate, allowing researchers across disciplines to confidently open the black box of machine learning.

Key Foundational Work

Predicting Student Dropout in Self-Paced MOOC Course Using Random Forest Model (2021) — contributed 1 concept elements to this direction.
Relevance-Based Feature Masking: Improving Neural Network Based Whale Classification Through Explainable Artificial Intelligence (2019) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_shap_feature_importance_bed91356,
  title = {SHAP feature importance},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/shap-feature-importance/rdr_bed91356c52b4d1b9e4fdb591824eb23},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_shap_feature_importance_bed91356_v1,
  title = {SHAP feature importance},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/shap-feature-importance/rdr_bed91356c52b4d1b9e4fdb591824eb23/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake