Improving Correlation Capture in Generating Imbalanced Data using Differentially Private Conditional GANs

Chang Sun, Johan van Soest, Michel DumontierarXiv 2022 · DOI · arXiv

Abstract

This paper introduces DP-CGANS, a framework designed to generate high-quality synthetic tabular data that preserves both complex correlations and individual privacy. By combining a novel conditional sampling strategy for imbalanced data with a differentially private training process, the authors demonstrate the ability to create "digital twins" of sensitive health datasets that are statistically accurate and safe for open research. Evaluation across five diverse datasets shows that DP-CGANS outperforms existing state-of-the-art models in capturing variable dependencies while maintaining high machine learning utility.

Introduction

Generative Adversarial Networks (GANs) have seen immense success in image and text generation, but tabular data—common in medical and socio-economic records—remains a significant challenge. The authors identify three primary hurdles in this domain:

  1. Imbalanced Variables: Real-world datasets often contain rare classes that standard GANs tend to ignore, failing to capture their specific relationships with other variables.
  2. Complex Correlations: Capturing non-linear dependencies (e.g., the relationship between age and health status) is difficult when data is sparse or non-Gaussian.
  3. Privacy Risks: GANs can inadvertently memorize training samples. If a model generates a synthetic record too similar to a real individual, it risks violating privacy regulations like GDPR or HIPAA.

Methodology

DP-CGANS addresses these challenges through a multi-layered architecture designed for stability and privacy.

Overview of the DP-CGANS architecture, illustrating the data transformation, sampling, and differentially private training pipeline.

Data Transformation and Normalization

To handle continuous variables, the model employs Mode-Specific Normalization. Instead of simple scaling, it uses a Variational Gaussian Mixture Model to estimate underlying distributions, effectively managing multi-modal data (e.g., height distributions with distinct peaks for different genders).

The Pairwise Conditional Vector

A core innovation of DP-CGANS is its conditioning strategy. To capture dependencies, the model randomly pairs two categorical variables and constructs a conditional vector. Using a log-probability sampling method, it ensures that rare combinations are presented to the model frequently enough during training to be learned accurately.

Detailed view of the conditional vector mechanism and the training loop involving the generator and discriminator.

Training and Privacy

The framework utilizes WGAN-GP (Wasserstein GAN with Gradient Penalty) for training stability and PacGAN to prevent mode collapse by having the discriminator evaluate "packs" of samples. Privacy is enforced by injecting Gaussian noise into the discriminator's gradients. By applying Rényi Differential Privacy (RDP), the authors provide a mathematical guarantee on the privacy budget (ε) spent, ensuring individual records cannot be re-identified.

Experimental Results

The model was evaluated on five datasets, including public benchmarks (Adult, Census, Intrusion) and real-world health data (Diabetes, Cancer).

  • Superior Correlation Capture: DP-CGANS outperformed models like CTGAN and TableGAN in capturing dependencies, particularly on complex datasets like Census and Diabetes.
  • Machine Learning Utility: Models trained on DP-CGANS synthetic data achieved performance metrics (AUC and F1) nearly identical to those trained on real data. For instance, on the Adult dataset, Logistic Regression achieved an AUC of 0.818 with synthetic data versus 0.820 with real data.
  • Privacy-Utility Trade-off: While increasing privacy (lowering ε) naturally reduces statistical similarity, DP-CGANS maintains a superior Pareto front compared to previous differentially private generators.

Conclusion and Future Work

DP-CGANS represents a significant step forward in generating private, high-fidelity synthetic tabular data. However, limitations remain, such as the "dimensionality explosion" that would occur if conditioning on more than two variables simultaneously. Future research may explore applying differential privacy directly to the generator or testing the framework on larger-scale "Big Data" environments to further validate its scalability.

Summary updated

Abstract

Despite the remarkable success of Generative Adversarial Networks (GANs) on text, images, and videos, generating high-quality tabular data is still under development owing to some unique challenges such as capturing dependencies in imbalanced data, optimizing the quality of synthetic patient data while preserving privacy. In this paper, we propose DP-CGANS, a differentially private conditional GAN framework consisting of data transformation, sampling, conditioning, and networks training to generate realistic and privacy-preserving tabular data. DP-CGANS distinguishes categorical and continuous variables and transforms them to latent space separately. Then, we structure a conditional vector as an additional input to not only presents the minority class in the imbalanced data, but also capture the dependency between variables. We inject statistical noise to the gradients in the networking training process of DP-CGANS to provide a differential privacy guarantee. We extensively evaluate our model with state-of-the-art generative models on three public datasets and two real-world personal health datasets in terms of statistical similarity, machine learning performance, and privacy measurement. We demonstrate that our model outperforms other comparable models, especially in capturing dependency between variables. Finally, we present the balance between data utility and privacy in synthetic data generation considering the different data structure and characteristics of real-world datasets such as imbalance variables, abnormal distributions, and sparsity of data.
BibTeX
@misc{sun2022improving,
  author = {Chang Sun and Johan van Soest and Michel Dumontier},
  title = {{Improving Correlation Capture in Generating Imbalanced Data using Differentially Private Conditional GANs}},
  year = {2022},
  eprint = {2206.13787},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2206.13787},
}
Figure