MargCTGAN: A "Marginally" Better CTGAN for the Low Sample Regime

Tejumade Afonja, Dingfan Chen, Mario Fritz

TL;DR

Generating high-quality synthetic tabular data is notoriously difficult when training data is scarce. This paper introduces MargCTGAN, an enhancement to the popular CTGAN model that incorporates a moment-matching loss on decorrelated features (via PCA) to significantly improve performance and statistical fidelity in low-sample scenarios.

What Problem Does This Paper Address?

Synthetic tabular data is a cornerstone for privacy-preserving data sharing and mitigating class imbalance. However, most state-of-the-art (SOTA) generators, like CTGAN, are "data hungry." When the available training set is small—the "low-sample regime"—the statistical properties of the generated data deteriorate rapidly.

Existing evaluation methods often focus solely on "downstream utility" (how well a model performs when trained on synthetic data). The authors argue this is insufficient. A good generator should also maintain fidelity (matching the real data's distribution) and diversity without simply memorizing the training set. The gap addressed here is the lack of a robust generator that maintains both statistical accuracy and utility when data is limited.

Methods

The authors build upon the CTGAN framework, which uses a Conditional GAN architecture with mode-specific normalization to handle heterogeneous tabular data. Their primary innovation, MargCTGAN, introduces a new loss term:

  1. Decorrelation via PCA: Before calculating statistics, the data is transformed using Principal Component Analysis (PCA). Crucially, they keep the original dimensionality but use PCA to decorrelate the features.
  2. Moment Matching Loss ($L_{marg}$): The generator is forced to match the first (mean) and second (standard deviation) moments of the real data within this decorrelated PCA space.
  3. Why this works: By decorrelating the marginals first, the moment-matching objective becomes more effective, as the model doesn't have to struggle with complex feature interdependencies while trying to get the basic "shape" of the individual columns right.

The total loss for the generator becomes a combination of the standard WGAN-GP loss, a conditional loss for categorical balance, and this new marginal matching loss.

Datasets

The researchers evaluated their model across four diverse benchmark tabular datasets:

  • Adult: A classic census dataset for income classification.
  • Census: A larger, more complex version of the census data.
  • News: A dataset for predicting the popularity of online news articles.
  • Texas: A large public health dataset regarding hospital discharges.

To test the "low-sample regime," they trained the models on subsets of these data ranging from as few as 40 samples up to the full dataset size.

Key Insights and Ideas

  • The "Goldilocks" Zone of Fidelity: The paper reveals a fascinating trade-off. TVAE (a Variational Autoencoder approach) often achieves the highest utility but risks "privacy leakage" because its synthetic records are too close to the real ones. TableGAN, conversely, stays far from the real data (good for privacy) but suffers in utility. MargCTGAN finds a middle ground, offering better utility than CTGAN without the extreme overfitting risks of TVAE.
  • The Failure of CTGAN in Sparsity: Standard CTGAN shows a sharp decline in performance once the training set drops below ~640 samples. MargCTGAN effectively "smooths" this decline, making it much more robust for small-scale applications.
  • Categorical vs. Numerical Strengths: The authors found that while TVAE excels at numerical data, it struggles with categorical columns. CTGAN is the opposite. MargCTGAN manages to balance both, matching CTGAN’s categorical performance while significantly boosting numerical accuracy.

Key Results

  • Utility Boost: In low-sample settings, MargCTGAN consistently outperformed the baseline CTGAN. For example, CTGAN exhibited relative errors in machine learning efficacy of up to 57% in sparse settings, which MargCTGAN significantly reduced.
  • Statistical Alignment: MargCTGAN showed superior "Histogram Intersection" scores, meaning the distribution of individual columns in the synthetic data much more closely matched the real data compared to the baseline.
  • Correlation Preservation: Even though the new loss focuses on marginals, the model followed the "real data reference" trend for pairwise column correlations more closely than other models in low-resource settings.

Limitations and Future Work

The authors identify a specific mathematical bottleneck: Rank Deficiency. The PCA-based moment matching relies on the relationship between the number of samples ($N$) and the number of features ($D$). When $N < D$ (extremely low samples relative to the number of columns), the PCA transformation becomes less competitive because it cannot accurately capture all feature characteristics.

Future work could involve exploring alternative decorrelation methods or adaptive loss weights that shift as the sample size changes. Additionally, while this paper focuses on fidelity and utility, a deeper dive into formal differential privacy guarantees for MargCTGAN remains an open area for exploration.

Summary updated

Abstract

The potential of realistic and useful synthetic data is significant. However, current evaluation methods for synthetic tabular data generation predominantly focus on downstream task usefulness, often neglecting the importance of statistical properties. This oversight becomes particularly prominent in low sample scenarios, accompanied by a swift deterioration of these statistical measures. In this paper, we address this issue by conducting an evaluation of three state-of-the-art synthetic tabular data generators based on their marginal distribution, column-pair correlation, joint distribution and downstream task utility performance across high to low sample regimes. The popular CTGAN model shows strong utility, but underperforms in low sample settings in terms of utility. To overcome this limitation, we propose MargCTGAN that adds feature matching of de-correlated marginals, which results in a consistent improvement in downstream utility as well as statistical properties of the synthetic data.
BibTeX
@inproceedings{afonja2023margctgan,
  author = {Tejumade Afonja and Dingfan Chen and Mario Fritz},
  title = {{MargCTGAN: A "Marginally" Better CTGAN for the Low Sample Regime}},
  booktitle = {DAGM},
  year = {2023},
  doi = {10.1007/978-3-031-54605-1\_34},
}

Loading related content...

Figure