Preservation of Anomalous Subgroups On Machine Learning Transformed Data

Samuel C. Maina, Reginald E. Bryant, William O. Goal, Robert-Florian Samoilescu, Kush R. Varshney, Komminist WeldemariamarXiv 2019 · arXiv

Introduction

In the era of big data, the tension between data privacy and data utility remains a central challenge. Researchers often need to identify "anomalous subgroups"—specific segments of data where machine learning models exhibit failure or bias. However, privacy regulations like GDPR and HIPAA restrict access to the raw data necessary for such audits. This paper introduces the Utility Guaranteed Deep Privacy (UGDP) system, which leverages Variational Autoencoders (VAEs) to generate anonymized synthetic data that preserves critical statistical patterns for subgroup discovery while protecting individual identities.

Problem Statement

Identifying subgroups such as patients with rare drug reactions or high-risk financial customers is essential for responsible AI. Traditional anonymization techniques, such as masking or blurring, often disrupt the complex statistical relationships required for machine learning. The fundamental question addressed here is: Can a dataset be transformed to prevent individual re-identification while ensuring that machine learning models still encounter the same systematic biases present in the original data?

The UGDP Framework

The proposed UGDP pipeline consists of four primary stages designed to balance privacy and utility:

  1. Bias Discovery: Utilizing the Bias-Scan algorithm, the system searches for subgroups where a classifier’s predicted odds ratio deviates significantly from observed outcomes, rather than relying on pre-defined demographic categories.
  2. Data Synthesis: A Variational Autoencoder (VAE) learns the latent structure of the sensitive data to generate entirely new synthetic records.
  3. Utility Evaluation: New models (Random Forest and XGBoost) are trained on this synthetic data to re-run the Bias-Scan.
  4. Comparison via Jaccard Distance: The system measures privacy (overlap of individual records) and utility (overlap of defining attribute values) between the original and synthetic anomalous subgroups.
The UGDP pipeline for bias discovery and data synthesis.

Experimental Results

The UGDP system was evaluated using the Bank of Portugal marketing dataset and the UCI Adult Census dataset. The results demonstrate that privacy and utility are not necessarily a zero-sum game.

Model Performance and Privacy

Classifiers trained on synthetic data maintained high performance, with AUC scores closely matching those of models trained on original data. Crucially, the Jaccard distance for individual records remained near 1.0, indicating that the specific individuals flagged in the synthetic data were entirely different from those in the original set, thereby ensuring high privacy.

Preservation of Anomalies

While individual identities were protected, the nature of the anomalies was preserved. The Jaccard distance for attribute values hovered around 0.5, suggesting that the VAE successfully captured the "essence" of the biased subgroups.

Jaccard distance metrics for individual records and attribute values across synthetic datasets.

Discussion

The core finding is that a VAE can "scramble" individual identities while keeping the "shape" of data bias intact. This suggests that the most effective "figure of merit" for synthetic data utility in auditing contexts is the consistency of discovered anomalous subgroups rather than just overall model accuracy. The authors observed a "sweet spot" where individual record overlap is zero, yet the defining characteristics of the subgroups remain identifiable for researchers.

Conclusion and Future Directions

While UGDP shows promise, limitations remain. VAEs may struggle with fine-grained constraints between attributes (e.g., ensuring logical consistency in demographic fields). Future work could involve integrating more advanced generative models like Generative Adversarial Networks (GANs) and testing the framework on higher-dimensional datasets to ensure the preservation of bias holds across more complex feature spaces.

Summary updated

BibTeX
@misc{maina2019preservation,
  author = {Samuel C. Maina and Reginald E. Bryant and William O. Goal and Robert-Florian Samoilescu and Kush R. Varshney and Komminist Weldemariam},
  title = {{Preservation of Anomalous Subgroups On Machine Learning Transformed Data}},
  year = {2019},
  eprint = {1911.03674},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.03674},
}
Figure