Towards a Game-Theoretic Understanding of Explanation-Based Membership Inference Attacks

Kavita Kumari, Murtuza Jadliwala, Sumit Kumar Jha, Anindya Maiti

Introduction

Membership Inference Attacks (MIA) represent a critical privacy challenge in machine learning, where an adversary attempts to determine whether a specific data point was included in a model's training set. While traditional MIAs rely on prediction confidence scores, recent research has shifted toward "explanation-based" attacks. these exploit the outputs of Explainable AI (XAI) methods—such as saliency maps or feature attributions—by analyzing the variance in these explanations.

Despite the empirical success of these attacks, the field has lacked a rigorous theoretical foundation. Most existing studies treat MIA as a "one-shot" interaction and rely on heuristic thresholds to separate members from non-members. This paper addresses these limitations by modeling the interaction between an adversary and a system as a continuous-time stochastic signaling game, providing a formal mathematical proof for the existence of optimal attack thresholds.

The Limitations of Current MIA Research

The authors identify two primary gaps in the current understanding of explanation-based MIAs:

  1. The One-Shot Limitation: Most frameworks assume a single query interaction. In practice, an adversary can query a system multiple times, iteratively refining their strategy based on the feedback (explanations) received.
  2. Threshold Uncertainty: Attacks typically require a threshold to distinguish training data members. However, there has been no theoretical guarantee that an optimal threshold exists, nor a clear path for an adversary to find it without internal knowledge of the model.

A Game-Theoretic Framework

To bridge these gaps, the researchers apply concepts from economic theory and financial mathematics to the cybersecurity domain. They model the interaction as a two-player game between a "System" (the ML model and its XAI component) and an "End-user" (who may be honest or malicious).

Stochastic Signaling and GBM

The evolution of explanation variance over repeated queries is modeled using Geometric Brownian Motion (GBM). While GBM is traditionally used to model stock price fluctuations, here it captures how the informational "value" of an explanation changes over time. This allows the researchers to treat the attack as a dynamic process rather than a static event.

Optimal Control and Equilibrium

To determine the optimal strategies for both players, the authors employ Hamilton-Jacobi-Bellman (HJB) equations, a cornerstone of optimal control theory. They seek a Markov Perfect Equilibrium (MPE)—a state where neither the attacker nor the system can improve their outcome by unilaterally changing their strategy. Within this framework, the adversary plays a "stopping game," calculating the precise moment they have accumulated enough information (variance) to trigger the attack with maximum success and minimum risk of detection.

Experimental Evaluation

The framework was evaluated across several benchmarks, including:

  • Tabular Data: Purchase, Texas, and Adult census datasets.
  • Image Data: CIFAR-10 and CIFAR-100.
  • Synthetic Data: Used to isolate the effects of input dimensionality and model complexity.

Key Insights

  • Existence of Optimal Thresholds: The paper provides a mathematical proof that an optimal threshold exists. This transforms MIA from a heuristic guessing game into a theoretically grounded optimization problem.
  • The System’s Dilemma: The system faces a trade-off between utility and privacy. Adding noise to explanations can thwart attackers, but excessive noise renders the explanations useless for honest users. The game-theoretic model helps identify the equilibrium point for this noise injection.
  • Dataset Complexity: The attack's success varied by dataset. On the Texas and Purchase datasets, the adversary often achieved 100% accuracy once the optimal threshold was identified. Conversely, CIFAR-10 proved more resilient; the high natural fluctuations in its explanation variance made it difficult for the attacker to converge on a stable equilibrium.
  • Beyond Overfitting: While overfitting is a known driver of MIA vulnerability, the authors found that it is not the only factor. The specific XAI method used (e.g., Integrated Gradients vs. LRP) and the model's internal structure are equally critical in determining "leakiness."

Conclusion and Future Work

This research moves the study of explanation-based MIAs into a formal mathematical territory. By proving that adversaries can "inch closer" to a successful attack through repeated interactions, the authors highlight the need for more dynamic defense mechanisms.

Future work remains to be done regarding the assumptions of the model. Currently, the framework assumes "risk-neutral" players and focuses primarily on gradient-based XAI methods. Expanding this to risk-averse scenarios and perturbation-based methods like SHAP or LIME will be essential for a comprehensive understanding of ML privacy in the wild.

Summary updated

BibTeX
@inproceedings{kumari2024towards,
  author = {Kavita Kumari and Murtuza Jadliwala and Sumit Kumar Jha and Anindya Maiti},
  title = {{Towards a Game-Theoretic Understanding of Explanation-Based Membership Inference Attacks}},
  booktitle = {GameSec},
  year = {2024},
  doi = {10.1007/978-3-031-74835-6\_13},
}
Figure