Synthetic tabular data is usually judged by how closely it matches the real table it was learned from. When that real table is dirty, with mislabeled rows, corrupted measurements, or values missing for systematic reasons, this yardstick fails without warning. A generator that faithfully copies the dirt scores as "high fidelity," and its training loss looks just as healthy as on clean data. A downstream model trained on its output then does worse in deployment. We propose to measure a generator's tendency to copy corruption directly, using the trick biologists call a spike-in. We add a small, known dose of artificial corruption to the already-dirty training table in a region we choose, retrain the generator, and measure how much of that known dose shows up in its synthetic output. Because we planted the corruption ourselves, the measurement needs no clean reference data, which practitioners almost never have.
Put in two sentences: you cannot tell from dirty data alone whether a generator is reproducing the dirt, but you can tell whether it reproduces dirt you planted yourself. If the planted dirt resembles the native dirt, the rate at which a generator passes the planted dirt into its synthetic data estimates how much of the native dirt it is passing on, and that rate should rank generators by real downstream usefulness better than fidelity scores do.
Core Claim
For each generator (and each setting of it), define a corruption transfer coefficient. Let $\varepsilon$ be the rate of probe corruption injected into a chosen region $R$ of feature space, and let $\Delta(\varepsilon)$ be the excess corruption measured inside $R$ in the synthetic data, relative to a run with no probe. The coefficient is the slope
$$ \tau = \frac{d\,\Delta(\varepsilon)}{d\,\varepsilon}, $$
estimated by regression over a few probe doses. A generator that memorizes noise has $\tau$ near 1. A generator that smooths noise toward the dominant pattern has $\tau$ near 0. We claim three things. First, $\tau$, computed from dirty data alone, ranks generators by downstream utility on clean held-out data better than divergence and detection scores, better than all-column structural scores computed against the dirty table, and better than the "easy-sample proportion" profiling signal. Second, $\tau$ explains when tree and Bayesian generators beat diffusion generators: under heavy noise and small samples, the winner is whichever transfers less. Third, the probe has to resemble the native corruption. Probes placed according to example hardness should predict utility well, while uniformly random probes should do noticeably worse. This is the ingredient we isolate.
Relation to Prior Work
The motivating evidence comes from two papers. An empirical study on impact of label noise on synthetic tabular data generation injects random and instance-dependent label flips (0–50%) into UCI Adult and shows that loss curves for CTGAN, TVAE, and CopulaGAN converge identically regardless of noise. That study measures damage using a clean copy of the data. We use injection as a measuring instrument on data that is already dirty, with no clean copy. Beyond Statistical Fidelity shows low total-variation distance alongside zero ontological validity on tiny EHR cohorts. Its fix depends on a curated medical ontology, which our probes do not need.
The closest competitors are evaluation methods. Reimagining Synthetic Tabular Data Generation through Data-Centric AI profiles samples as easy, ambiguous, or hard using Cleanlab, Data-IQ, and Data Maps. It filters on those profiles and finds that the proportion of easy synthetic samples predicts downstream AUC better than fidelity. This is the strongest prior signal. It scores the data, though, not the generator's propensity to transfer corruption, and its tests stop at 10% label noise. TabStruct introduces Global Utility, which predicts every column from all others and correlates with true causal-graph fidelity at $r_s = 0.84$, but only for generators trained on clean data. Computed against a dirty reference, it should reward copying the dirt, and we test that directly. Structured Evaluation of Synthetic Tabular Data and Synthetic Tabular Data Validation: A Divergence-Based Approach score agreement of conditionals and full joint distributions with the real table. That makes them precisely the metrics a dirt-copying generator would win.
Two papers constrain the design. Synthcity finds Adversarial Random Forests best on quality and detection, while TabStruct finds diffusion models best on structure. Our generator pool must include both families. IHCS shows rule-based cleaning works well when rules are complete, which tells us where our method is unnecessary.
Research Program
We will build probe families for four corruption types. Label probes flip labels. Feature probes add structured noise to one column. Missingness probes mask values with probability that depends on the value itself, so the data is missing not at random. Dependency probes shuffle one column within $R$, which breaks its relationship to the other columns. Each type comes in two placements. Random placements define $R$ by a random hyperplane. Hardness placements define $R$ as the examples a cross-fitted classifier finds hardest, following the instance-dependent noise construction of the label-noise study. To measure $\Delta$ in synthetic data, which has no row-to-row correspondence with training data, we use a classifier fit on held-out real rows, together with column statistics restricted to $R$. Doses are $\varepsilon \in \{0, 2, 5, 10\}\%$ of rows in $R$, with three seeds each.
The generator pool, run through Synthcity, is CTGAN, TVAE, a Bayesian network, ARF, synthpop, TabDDPM, and TabSyn. For each generator we also vary training length and capacity, because memorization depends on both. The main cost is retraining each generator about 12 times. We will report that cost against Global Utility, which trains $p$ ensembles. We will also test whether a single dose plus the zero-dose run, which needs only two fits, recovers the slope.
Baselines
The naive reference point is to train the downstream model on the dirty real table with no synthetic data. A second naive point is to pick the generator with the best Synthcity detection or fidelity score computed against the dirty table. Every proposal must at least beat this second rule, because it is what practitioners do.
The strongest existing competitor is the data-centric profiling pipeline of Reimagining Synthetic Tabular Data Generation. We use it two ways: as a selection score, ranking generators by the easy-sample proportion of their output, and as a filter applied post hoc. It reports classification gains up to +1.64% (13 of 15 evaluations improved) and model-selection ranking gains of 4.66% to 92%. It is the strongest competitor because it is also reference-free and was designed for this problem. We will run it with the same generators, the same dirty tables, and the same profiling classifier family as our hardness probes, with matched hyperparameter search. We will also give it compute equal to the probe retrains, for example by ensembling profilers over seeds. Beating it means a higher Spearman correlation between the selection score and clean-test downstream AUC, aggregated across datasets and noise levels, with a paired bootstrap confidence interval that excludes zero. We also require higher clean-test AUC for the generator each method selects.
The second strong competitor is TabStruct Global Utility in its Tiny-default ensemble form, computed against the dirty table. On clean data its reported rank correlation with true graph fidelity is $r_s = 0.84$. We will check whether that correlation survives on dirty data and whether $\tau$ beats it there. Two ablations isolate the key ingredients. Random-placement probes test whether resemblance to native noise matters. A single-dose estimate tests whether the slope matters or a cheap one-shot measurement suffices. A third ablation adds $\tau$ to the easy-proportion score, to check whether the two are complementary rather than redundant.
Why not just…?
Why not just use the easy-sample proportion from data profiling? This is the real threat. Hard-looking synthetic samples may already reveal copied label noise. Our case against it rests on two points that the experiments will check. First, profiling scores the output and cannot tell a generator that copies native noise from one that invents its own errors. Second, profiling is defined around labels and has nothing to say about missingness or broken dependencies among non-target columns. If easy-proportion matches $\tau$ on label noise, we will say so and limit our claim to feature, missingness, and dependency corruption, where the head-to-head comparison decides the matter.
Why not just compute Global Utility or a divergence score? These measure agreement with the reference table. When the reference is dirty, perfect copying of the dirt is perfect agreement, and the label-noise study's smooth loss curves show nothing in training flags it. Our controlled-dirt experiments, where a clean twin exists for scoring, measure directly how far Global Utility's ranking drifts from clean-test utility as noise grows.
Why not just clean the data first? IHCS reaches an F1-score of 0.9834 on Medicare data, but it depends on human-specified rules. Instance-dependent noise of the kind studied in An empirical study on impact of label noise is not captured by simple rules. We include Cleanlab-based removal before generation as an arm in the experiments. If cleaning plus any generator matches $\tau$-based selection, the probe is not needed. If $\tau$ still separates generators after cleaning, residual noise matters.
Decisive Experiments
The main test uses tables where truth is recoverable. These are UCI Adult, the 11 OpenML datasets from the data-centric benchmark, and the six expert-built causal-model datasets from TabStruct, including Hailfinder and Insurance. For each, we create a hidden "native" corruption at 5–40%, of each of the four types, at sample sizes from 100 to the full table. All selection scores see only the dirty table. Ground truth is downstream AUC on a clean test split, plus completed-partial-DAG fidelity on the causal-model datasets. The claim fails if $\tau$ does not beat both easy-proportion and dirty-reference Global Utility in rank correlation outside the low-noise regime. It also fails if hardness probes do no better than random probes, because that would mean the transfer measured on probes says nothing about native noise.
A secondary analysis plots, for each generator family, clean-test utility and $\tau$ against sample size and noise rate. This tests whether the conflicting rankings in Synthcity, TabStruct, and DataSynK reduce to differences in transfer. Finally, on tables with genuinely native problems, namely SDV student/expedia/census (natural missingness) and SIVEP-Gripe, where Synthcity reports TabDDPM raising minority-group AUROC from 0.7327 to 0.7786 with XGBoost, we will report whether $\tau$-based selection preserves or improves those gains. No clean ground truth exists on these tables, so they serve as a plausibility check, not proof.
References
- An empirical study on impact of label noise on synthetic tabular data generation
- Beyond Statistical Fidelity: Causal-Valid Synthetic EHR Generation for Low-Resource Clinical AI
- Reimagining Synthetic Tabular Data Generation through Data-Centric AI: A Comprehensive Benchmark
- TabStruct: Measuring Structural Fidelity of Tabular Data
- Structured Evaluation of Synthetic Tabular Data
- Synthetic Tabular Data Validation: A Divergence-Based Approach
- Synthcity: a benchmark framework for diverse use cases of tabular synthetic data
- IHCS: An Integrated Hybrid Cleaning System