Game-Theoretic Feature Attribution for Machine Learning

40 concepts40 papers2 proposals

Modern machine learning models can make incredibly accurate predictions, but they often operate as "black boxes." A deep neural network or a gradient-boosted tree might correctly flag a patient's risk of infection or identify a malicious botnet, but it cannot inherently explain why it arrived at that conclusion.

To trust these models in high-stakes domains like medicine, law, and cybersecurity, we need a way to look inside. The most prominent framework for doing this is game-theoretic feature attribution. By treating a model's prediction as a cooperative game where the input features are players, this approach provides a mathematically rigorous way to distribute "credit" for a prediction across its inputs.

The Problem with Heuristic Explanations

To understand why game theory is necessary, consider a simpler approach to explaining a model: feature ablation. If you want to know how important a patient's age is to an insurance risk model, you could simply remove the "Age" feature, re-run the model, and see how much the prediction changes.

This heuristic fails when features interact or contain redundant information. Imagine two features: "Age" and "Years since birth." If you remove "Age," the model can easily rely on "Years since birth" to make the exact same prediction. The change in output would be zero, misleadingly suggesting that age has no impact.

Conversely, if two features only matter when they appear together (a non-linear interaction), removing one at a time will obscure their joint importance. Heuristic methods lack a principled way to handle these dependencies, often leading to unstable or ungrounded explanations.

Framing Predictions as Cooperative Games

Game-theoretic feature attribution solves this by reframing the explanation problem entirely.

Imagine a cooperative game where a team of players works together to achieve a score. In the context of machine learning:

  • The players are the individual input features (e.g., blood pressure, age, glucose levels).
  • The game is the machine learning model's prediction function.
  • The payout is the difference between the model's specific prediction for an instance and its average (baseline) prediction across the entire dataset.

To fairly divide this payout among the features, we calculate their marginal contributions. Instead of evaluating a feature in isolation, we look at how the model's prediction changes when that feature is added to various subsets (coalitions) of other features.

As Amit Datta et al. demonstrated in Influence in Classification via Cooperative Game Theory, this coalitional framework allows us to measure the true influence of an input variable without needing to alter or understand the internal parameters of a black-box classifier. By averaging a feature's marginal contribution across all possible subsets of features, we obtain its Shapley value—a metric that quantifies exactly how much that feature shifted the prediction away from the baseline.

The Power of Axioms

Why should we trust Shapley values over other local explanation methods? The answer lies in their axiomatic foundation. As noted by Andreas Holzinger et al. in Explainable AI Methods - A Brief Overview, coalitional game theory is the only approach that satisfies a set of fundamental fairness properties:

  1. Efficiency (Local Accuracy): The sum of the attributions assigned to each feature must exactly equal the difference between the model's prediction and the baseline prediction. No explanation value is "lost" or unaccounted for.
  2. Symmetry: If two features contribute identically to all possible subsets, they receive the exact same attribution score.
  3. Dummy (Null Player): If a feature never changes the prediction when added to any subset, its attribution score is zero.
  4. Additivity: If a prediction is the sum of two distinct model outputs, the feature attributions must also sum together.

Because of these properties, game-theoretic attributions offer a level of mathematical consistency that heuristic alternatives cannot match. For instance, Isaac Ronald Ward et al. observed in Explainable artificial intelligence for pharmacovigilance: What features are important when predicting adverse outcomes? that game-theoretic methods provide significantly higher stability across different model runs compared to local surrogate models, which can be highly sensitive to minor data perturbations.

Overcoming the Combinatorial Explosion

The primary catch to calculating exact Shapley values is computational complexity. Because you must evaluate a feature's marginal contribution across every possible subset, the number of required model evaluations scales exponentially ($2^N$) with the number of features ($N$). For a model with dozens of inputs, exact calculation is impossible.

To make game-theoretic attribution practical, researchers focus on clever approximations and domain-specific optimizations:

From Theory to High-Stakes Applications

By converting complex statistical patterns into a fair distribution of feature contributions, game-theoretic attribution has become a cornerstone for deploying machine learning in critical fields.

Clinical Risk and Radiomics

In medicine, understanding why a model flags a patient can dictate treatment paths. Ting Peng et al. applied additive feature attribution to explain an XGBoost model predicting infection risks in multiple myeloma patients (Machine learning-based infection prediction model for newly diagnosed multiple myeloma patients). Similarly, Xingwang Peng et al. used the SHAP framework to decompose risk scores for feeding intolerance in preterm newborns into actionable clinical insights (Construction and SHAP interpretability analysis of a risk prediction model for feeding intolerance in preterm newborns based on machine learning).

This transparency is equally vital when models interpret high-dimensional medical imaging. In radiomics studies—such as differentiating solid renal tumors (Yaohai Wu et al.) or predicting breast cancer metastasis (Dingyi Zhang et al.)—game-theoretic attribution allows clinicians to verify that a model is focusing on legitimate biological markers (like geometric tumor flatness or functional MRI sequences) rather than capturing imaging artifacts.

Auditing and Vulnerability Detection

Game-theoretic attribution also serves as an auditing tool to uncover hidden model vulnerabilities. Dora Kaufman highlights these methods as a vital mechanism for identifying and mitigating algorithmic bias within deep learning systems (Viés no aprendizado de máquina em sistemas de inteligência artificial: a diversidade de origens e os caminhos de mitigação).

In cybersecurity, Giorgio Piras et al. used SHAP values to audit botnet detectors in Explaining Machine Learning DGA Detectors from DNS Traffic Data. Their analysis revealed that despite high accuracy, the models relied heavily on Time-to-Live (TTL) network features—a dangerous vulnerability that attackers could easily manipulate to evade detection.

By framing machine learning interpretation as a problem of fair distribution, game-theoretic feature attribution bridges the gap between mathematical complexity and human trust. It ensures that when a model makes a decision, we can precisely trace how every piece of data contributed to the final outcome.

Key Foundational Work

A Feature Selection Method Based on Shapley Value to False Alarm Reduction in ICUs, A Genetic-Algorithm Approach (2018) — contributed 1 concept elements to this direction.
Influence in Classification via Cooperative Game Theory (2015) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_game_theoretic_feature_attribution_af8f42c9,
  title = {game-theoretic feature attribution},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/game-theoretic-feature-attribution/rdr_af8f42c9605045308729e303ba19dc00},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_game_theoretic_feature_attribution_af8f42c9_v1,
  title = {game-theoretic feature attribution},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/game-theoretic-feature-attribution/rdr_af8f42c9605045308729e303ba19dc00/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake