Introduction
Data collaboration in Data Clean Rooms (DCRs)—environments where parties like advertisers and publishers share data without exposing raw sensitive information—is often hindered by complex data structures. While synthetic data serves as a privacy-preserving "mathematical twin" of real data, current generators struggle with many-to-many relationships common in collaborative datasets.
This paper introduces DEREC-SIMPRO, a framework designed to restructure multi-table data for advanced synthesizers and evaluate the results using rigorous, shape-sensitive metrics.
The Challenge: Architectural Mismatch and Evaluation Bias
The authors identify two primary roadblocks in synthetic data generation for DCRs:
- Architectural Mismatch: Advanced multi-table synthesizers, particularly those leveraging Large Language Models (LLMs), typically require a strict one-to-many (parent-child) hierarchy. Real-world collaborative data is often many-to-many, leading to poor synthesis quality.
- Evaluation Bias: Standard industry metrics (e.g., SDMetrics) often favor "smoothed" distributions. This can result in high scores for models that miss the unique, jagged "shape" of the original data, failing to capture true statistical utility.
Methodology: The DEREC-SIMPRO Framework
The proposed solution consists of two integrated components: the DEREC pipeline for data restructuring and the SIMPRO framework for evaluation.
1. The DEREC Pipeline (DEtect, REcreate, Connect)
DEREC acts as a pre-processing bridge to transform incompatible many-to-many data into a synthesizer-friendly one-to-many format:
- DEtect: Automatically identifies "contextual" columns (e.g., age, gender) that remain consistent for a user.
- REcreate: Constructs an artificial "parent table" where each subject appears once, populated by contextual columns.
- Connect: Links the new parent table to the original tables (now "child tables"), establishing the hierarchy required by LLM-based synthesizers.
2. The SIMPRO Evaluation Framework
SIMPRO addresses "smoothing bias" by evaluating data across three dimensions:
- Statistical Similarity: Uses the Kolmogorov-Smirnov (KS) test to ensure synthetic data captures overall table logic.
- IMprovement Counts: Provides a granular analysis of how many specific column pairs improved compared to baseline models.
- PRObabilistic Distance: Employs Wasserstein Distance to measure the literal "cost" of transforming synthetic distributions into real ones.
Key Insights and Results
The study utilized the CTR Prediction - 2022 DIGIX Global AI Challenge dataset, a real-world marketing dataset with complex relational structures.
Engineering Hierarchy
A key insight is that natural hierarchies are not a prerequisite for multi-table synthesizers. By algorithmically "inventing" a parent table through DEREC, the power of LLMs (like GPT-2) can be unlocked for previously incompatible datasets.
Shape Fidelity over Smoothness
The authors demonstrate that capturing the "true" distribution shape is more critical than achieving a smooth curve. SIMPRO prioritizes this "shape fidelity," identifying cases where traditional metrics might reward inaccurate but smooth distributions.
Performance Gains
- Fidelity Boost: The DEREC pipeline consistently improved synthetic data quality. In tests, DEREC-REaLTabFormer showed 147 improved column pairs compared to only 32 that worsened.
- Superiority over Baselines: The DEREC-enhanced model significantly outperformed single-table benchmarks like CT-GAN and TabDDPM, which failed to capture the collaborative data's underlying logic.
Limitations and Future Work
While effective, the current DEREC implementation synthesizes parent-child pairs independently, potentially missing correlations between different child tables. Future work includes developing a unified training approach and exploring more modern LLM backbones like GPT-4o or Llama 3 to further enhance synthesis quality.
Summary updated