Abstract
Valuation problems, such as feature interpretation, data valuation and model valuation for ensembles, become increasingly more important in many machine learning applications. Such problems are commonly solved by well-known game-theoretic criteria, such as Shapley value or Banzhaf value. In this work, we present a novel energy-based treatment for cooperative games, with a theoretical justification by the maximum entropy framework. Surprisingly, by conducting variational inference of the energy-based model, we recover various game-theoretic valuation criteria through conducting one-step fixed point iteration for maximizing the mean-field ELBO objective. This observation also verifies the rationality of existing criteria, as they are all attempting to decouple the correlations among the players through the mean-field approach. By running fixed point iteration for multiple steps, we achieve a trajectory of the valuations, among which we define the valuation with the best conceivable decoupling error as the Variational Index. We prove that under uniform initializations, these variational valuations all satisfy a set of game-theoretic axioms. We experimentally demonstrate that the proposed Variational Index enjoys lower decoupling error and better valuation performance on certain synthetic and real-world valuation problems.
Introduction
In machine learning, assigning "value" to individual components—be they features, training samples, or models in an ensemble—is crucial for interpretability and performance optimization. Cooperative game theory provides a rigorous mathematical foundation for this task, treating components as "players" and model outputs as "payouts." However, classical solutions like the Shapley and Banzhaf values often struggle with the complex, non-linear correlations between players in modern ML systems. This paper reinterprets these classical metrics through the lens of Energy-Based Models (EBMs), revealing them to be specific steps in a broader optimization process.
Methodology: Valuation as Decoupling
The authors propose a probabilistic treatment of cooperative games using the Maximum Entropy Principle. By representing a game as a probability distribution where the likelihood of a coalition forming is proportional to its payoff, they create an Energy-Based Model. To make this EBM tractable, the authors employ mean-field variational inference, which approximates the complex distribution with a factorized one where players are independent.
A key discovery is that the Banzhaf and Shapley values emerge as the result of a single step of fixed-point iteration in this optimization. By iterating to convergence ($K$ steps), the authors derive the Variational Index, which offers a more accurate "decoupling" of player interactions while still satisfying fundamental game-theoretic axioms like the null player and symmetry rules.
Experimental Evaluation
The Variational Index was tested across synthetic games, data valuation tasks, and feature attribution. The model includes a temperature parameter ($T$) that allows tuning the "fairness vs. concentration" tradeoff, where high $T$ spreads importance and low $T$ concentrates it on the most vital players.
Data Valuation
In data valuation experiments using the Breast Cancer and Digits datasets, the goal is to identify the most influential training samples. By removing data points in order of their assigned value, researchers can observe how quickly model accuracy degrades.
As shown in Figure 2, the Variational Index often leads to a faster decline in test accuracy compared to traditional metrics, indicating a superior ability to identify truly critical data points. Despite being an iterative method, it typically converges in just 5–10 steps, maintaining computational practicality comparable to calculating the Banzhaf value.
Conclusion
This work reinterprets classical game-theoretic valuations as a specific optimization process within an EBM framework. The resulting Variational Index provides a principled way to improve upon the Shapley and Banzhaf values by further decoupling player correlations. This approach not only offers better performance in data and feature valuation but also provides a new probabilistic perspective on the foundations of cooperative game theory in machine learning.
Summary updated