TL;DR
This paper introduces CG-TGAN, a novel conditional generative adversarial network (CGAN) that leverages Graph Neural Networks (GNNs) to synthesize realistic tabular data. By treating tabular data as a graph, CG-TGAN effectively captures complex inter-column relationships, outperforming existing GAN-based methods and showing comparable results to diffusion models in machine learning utility while offering superior privacy protection.
What Problem Does This Paper Address?
The increasing reliance on AI across various fields necessitates robust data sharing. However, sharing sensitive tabular data poses significant privacy risks. Synthesizing realistic tabular data is a crucial solution, but it's challenging due to the inherent complexity of tabular data, which often includes mixed data types, multimodal distributions, and imbalanced features. Existing GAN and diffusion models, typically using fully connected or convolutional layers, struggle to effectively model the intricate relational meanings between tabular columns. Fully connected layers lack inductive bias, while convolutional layers are sensitive to column order, which is semantically irrelevant. This paper aims to bridge this gap by proposing a GNN-based approach to better capture these relationships.
Methods
The core innovation of this paper is the CG-TGAN architecture, which adapts the GAN framework by incorporating Graph Neural Networks (GNNs).
- Tabular Data to Graph Conversion: Tabular data is transformed into a graph where each column becomes a node. The relationships between columns are represented by edges. To learn these relationships without explicit prior knowledge, a learnable parameter matrix ($A_p$) is introduced, which is then used to construct adjacency matrices for the generator ($A_G$) and discriminator ($A_D$).
- Conditional GAN Framework: CG-TGAN is a conditional GAN, meaning it generates data based on specific conditions. A novel "conditional node" is introduced into the graph structure. This node acts as a conduit for a conditional vector, allowing the GAN to generate data conditioned on specific attributes, similar to how virtual nodes enhance graph classification.
- Dual-Task Learning with GNNs: Both the generator and discriminator employ GNNs (specifically, Graph Convolutional Networks - GCNs) and are trained on two types of tasks:
- Graph-Level Task: Distinguishing between real and synthetic graphs (discriminator) or generating a graph that is macroscopically similar to real data (generator). This is achieved using a WGAN-GP loss.
- Node-Level Task: Predicting the value of a target node (discriminator) or synthesizing a graph with correct inter-node relationships (generator). This task is crucial for capturing nuanced column dependencies and is trained using cross-entropy or mean squared error loss depending on the column type.
- Data Encoding: For input, continuous columns are encoded using mode-specific normalization, mixed-type columns use a mixed-type encoder, and categorical columns are one-hot encoded. These encoded features are then projected to a common dimension to form the node feature matrix.
Datasets
The paper evaluates CG-TGAN on seven widely used datasets:
- Regression: Abalone, King, Insurance
- Binary Classification: Adult, Diabetes, Wilt
- Multi-class Classification: Gesture
These datasets are used for both training and evaluating the performance of CG-TGAN against various baselines. No new benchmarks are introduced.
Key Insights and Ideas
- Graph Representation for Tabular Data: The fundamental insight is that treating tabular data as a graph, where columns are nodes and relationships are edges, provides a more intuitive and effective way to capture inter-column dependencies than traditional fully connected or convolutional layers.
- Conditional Node for Enhanced Conditioning: The introduction of a "conditional node" is a clever way to integrate conditional information into the GNN-based GAN, improving its ability to generate data under specific constraints.
- Dual-Task Learning for Comprehensive Synthesis: Training both the generator and discriminator on graph-level and node-level tasks simultaneously allows for a more holistic approach to data synthesis, ensuring both overall data distribution and fine-grained relationships are captured.
- GNNs as a Superior Alternative: The success of CG-TGAN validates the hypothesis that GNNs are a powerful tool for tabular data synthesis, offering advantages in capturing complex relationships that are missed by other architectures.
Key Results
- Outperforms GAN Baselines: CG-TGAN consistently outperforms existing GAN-based tabular data synthesizers (DPGAN, Pate-GAN, CTGAN, CTAB-GAN, CTAB-GAN+) in terms of machine learning utility across both regression and classification tasks.
- Comparable to Diffusion Models: CG-TGAN achieves comparable machine learning utility to diffusion-based models like CoDi and TabDDPM.
- Superior Privacy Protection: CG-TGAN demonstrates better privacy protection than TabDDPM, as indicated by higher Nearest Neighbor Distance Ratio (NNDR) and Distance to Closest Record (DCR) values across most datasets.
- Effective for Small Datasets: CG-TGAN shows dominance over TabDDPM for small tabular datasets (under 1000 rows) in terms of machine learning utility.
Limitations and Future Work
- Ablation Study on GNN Layers: While the paper explores the number of GNN layers, it notes that the optimal number can vary, and further investigation into more sophisticated GNN architectures could be beneficial.
- Computational Cost: While not explicitly detailed as a limitation, the use of GNNs and dual-task training might introduce higher computational overhead compared to simpler GAN architectures.
- Future Work: The authors suggest that their work provides a strong foundation for future research in GNN-based tabular data synthesis, potentially exploring more advanced GNN variants or different graph construction methods. The success in privacy protection also opens avenues for further research in secure data synthesis.
Summary updated