Reconstructing Training Data from Diverse ML Models by Ensemble Inversion

Qian Wang, Daniel Kurz

Introduction

Model Inversion (MI) represents a critical privacy vulnerability in machine learning, where an adversary attempts to reconstruct sensitive training data—such as biometric faces or medical records—by querying a target model. While traditional MI attacks focus on a single model, they often yield low-fidelity, unrecognizable results. Researchers from Apple have introduced Ensemble Inversion, a novel framework that significantly enhances reconstruction accuracy by simultaneously attacking multiple correlated models trained on the same underlying data.

The Ensemble Inversion Framework

The core of the proposed method is a Generative Adversarial Network (GAN) architecture designed to synthesize images that satisfy the collective predictions of a group of Models Under Attack (MUAs). Unlike standard MI, which relies on a single feedback loop, Ensemble Inversion leverages the diverse perspectives of multiple models to provide a more comprehensive set of constraints for the generator.

The Ensemble Inversion architecture uses a GAN to generate samples that satisfy multiple Models Under Attack (MUAs) simultaneously.

Technical Innovations

To optimize the reconstruction process, the authors introduced several key components:

  • Ensemble Guidance: The GAN's generator is guided by an ensemble of models. This multi-model feedback forces the generator to produce samples that align with the features learned by all models in the set.
  • Specialized Loss Functions: The framework employs One-Hot Loss ($L_{OH}$) and Maximum Response Loss ($L_{MR}$). These functions encourage the generation of "sharp" images that elicit high-confidence predictions from the MUAs, preventing the blurry, ambiguous outputs typical of single-model attacks.
  • Farthest Model Sampling (FMS): To maximize information gain, the authors developed FMS to select the most diverse subset of available models. Greater model dissimilarity leads to more robust reconstructions.
  • Class Correspondence: A covariance-based method was implemented to align class labels across different models, ensuring that the attack targets the same identity across the entire ensemble.

Why Ensembles Work: The "Blind Men and the Elephant" Effect

The success of Ensemble Inversion stems from the fact that different models, even when trained on the same data, often focus on distinct features. This phenomenon is analogous to the "Blind Men and the Elephant" parable, where each model observes a different part of the whole.

Heatmaps demonstrate that different models (Model 1, 2, and 3) focus on different facial regions for the same input, providing complementary information for reconstruction.

As shown in the visualization above, one model might prioritize the eyes, while another focuses on the mouth or jawline. By attacking these models as an ensemble, the adversary can "stitch" these disparate features together, resulting in a reconstructed image that is far more recognizable and accurate than any single-model output.

Experimental Evaluation

The authors evaluated Ensemble Inversion across several datasets, demonstrating substantial improvements over baseline single-model attacks:

  • MNIST (Handwritten Digits): In a data-free setting (no auxiliary data), reconstruction accuracy improved by 70.9%. When auxiliary data (EMNIST) was provided, accuracy reached 97.9% with an ensemble of four models.
  • VGGFace2 (Biometric Data): Attacking an ensemble of five face classifiers resulted in a 21.1% increase in reconstruction accuracy. Qualitatively, the generated faces exhibited clear, identifiable structures that were absent in single-model inversions.

Conclusion

Ensemble Inversion highlights a significant and previously overlooked vulnerability in the deployment of multiple related ML models. By demonstrating that model diversity actually aids the attacker, this research underscores the need for more robust privacy-preserving techniques. Future work will focus on developing defense mechanisms, such as specialized differential privacy protocols, to mitigate the risks posed by multi-model ensemble attacks.

Summary updated

BibTeX
@inproceedings{wang2022reconstructing,
  author = {Qian Wang and Daniel Kurz},
  title = {{Reconstructing Training Data from Diverse ML Models by Ensemble Inversion}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
  doi = {10.1109/wacv51458.2022.00392},
}
Figure