Abstract
Inf2Guard is a unified information-theoretic framework designed to protect machine learning models from membership, property, and data reconstruction inference attacks. By optimizing mutual information objectives, it learns data representations that maximize task utility while providing provable privacy guarantees. Evaluation across diverse benchmarks demonstrates that Inf2Guard achieves a superior privacy-utility trade-off compared to traditional methods like Differential Privacy.
Introduction
Machine learning models are prone to "leaking" sensitive information about their training data. Even without direct access to the training set, adversaries can perform inference attacks to extract private details. These attacks generally fall into three categories:
- Membership Inference (MIA): Determining if a specific sample was part of the training set.
- Property Inference (PIA): Inferring global statistics of the training dataset.
- Data Reconstruction (DRA): Rebuilding original raw inputs from model representations.
Existing defenses are often siloed, targeting only one attack type, and frequently impose a significant "utility tax," degrading model accuracy. Inf2Guard proposes a principled, unified framework to defend against all three attack vectors simultaneously while maintaining high performance.
Methodology: Privacy-Preserving Representation Learning
Inf2Guard treats privacy as a representation learning problem. Instead of training a standard classifier, the framework trains an encoder $f_\theta$ to produce representations $r$ that satisfy two competing Mutual Information (MI) objectives:
- Privacy Objective: Minimize the MI between the representation $r$ and the private attribute $u$ (e.g., membership status).
- Utility Objective: Maximize the MI between the representation $r$ and the target task label $y$.
Since exact MI calculation is computationally intractable for deep neural networks, Inf2Guard utilizes variational MI bounds. Specifically, it employs vCLUB (Variational Contrastive Log-ratio Upper Bound) for MIA and PIA. For high-dimensional data reconstruction (DRA), it uses Jensen-Shannon Divergence (JSD) and introduces a learned randomized perturbator $\delta$ to ensure representations are non-deterministic, making them significantly harder to invert.
Experimental Evaluation
The authors evaluated Inf2Guard across several benchmarks, including CIFAR10, Purchase100, and Texas100 for MIA; Census and RSNA Bone Age for PIA; and Activity sensor data for DRA.
Key Results and Trade-offs
Inf2Guard consistently outperforms standard defenses like Differential Privacy (DP). For example, against MIA on the CIFAR10 dataset, Inf2Guard achieved 77% utility while keeping attack accuracy at 51% (near random guessing). In comparison, the standard DP-SGD defense required dropping utility to 48% to reach the same privacy level.
Furthermore, Inf2Guard provides the first effective defense against Property Inference, reducing PIA accuracy on the Census dataset from 68% to 34% without sacrificing task accuracy. In reconstruction attacks, images recovered from Inf2Guard-protected representations appear as unrecognizable "blobs," whereas DP-protected models often retain identifiable structures.
Conclusion and Future Work
Inf2Guard provides a unified mathematical language for defending against diverse inference attacks with formal theoretical guarantees. By leveraging sophisticated MI estimators and optimal noise distributions, it establishes a new state-of-the-art Pareto frontier for the privacy-utility trade-off. Future research directions include developing task-agnostic encoders that protect privacy without prior knowledge of the downstream utility task and optimizing the framework to defend against multiple attack types simultaneously.
Summary updated