When practitioners mix synthetic data into training, they want to know three things: how much real data the synthetic data can replace, how much synthetic data is worth adding, and which downstream metrics will suffer. Today each answer comes from brute force. You retrain the downstream model at many real-to-synthetic ratios and test each one on real data. This proposal argues that, for a large and practically important family of downstream models, the whole substitution curve can be predicted from one fit on synthetic data plus a small real sample. The prediction rests on a single geometric fact.
The models in question are those trained with a likelihood or other Bregman-type loss: logistic and linear regression, log-linear models of categorical tables, and the final classification layer of a deep network. For these models, each candidate model can be described by its "dual coordinates," the expected values of the statistics the model is fitted to match. Mixing two datasets moves the fitted model along a straight line in those coordinates. That is exactly the "dually flat" structure studied in information geometry. The cost of mixing in synthetic data is therefore a Bregman divergence along that line, a curved distance measured in the model's own Fisher metric. It depends only on how the synthetic distribution differs from the real one in the directions the downstream model actually uses. This explains why heavily distorted synthetic data sometimes helps and faithful data sometimes doesn't. It also turns the vague notion of an "exchange rate" into a quantity that can be computed and then checked.
Core Claim
Let $P$ be the real distribution and $Q$ the synthetic one. Let the downstream model belong to an exponential family with sufficient statistics $T$ and log-partition (potential) function $F$. Its Legendre conjugate $F^*$ defines the dual coordinates $\eta = \mathbb{E}[T]$.
Training on a mixture with synthetic fraction $\alpha$ yields, at the population level, the model whose dual coordinate is $\eta_\alpha = (1-\alpha)\eta_P + \alpha\eta_Q$. Its excess risk on real data is exactly
$$ \mathcal{E}(\alpha) = B_{F^*}\big(\eta_P : \eta_P + \alpha\,\Delta\eta\big), \qquad \Delta\eta = \mathbb{E}_Q[T] - \mathbb{E}_P[T], $$
where $B_{F^*}$ is the Bregman divergence generated by the conjugate potential. To second order this is $\tfrac{1}{2}\alpha^2\,\Delta\eta^\top G^{-1}\Delta\eta$, with $G$ the Fisher information of the downstream model at the real optimum.
Adding finite-sample variance gives a predicted real-data risk for $n_r$ real and $n_s$ synthetic samples:
$$ \mathcal{E}(n_r, n_s) \approx \tfrac{1}{2}\alpha^2\,\Delta\eta^\top G^{-1}\Delta\eta + \frac{d_{\mathrm{eff}}}{2(n_r+n_s)}, \qquad \alpha = \frac{n_s}{n_r+n_s}, $$
where $d_{\mathrm{eff}}$ is the trace of $G^{-1}$ times the score covariance under the mixture. This variance form is the standard asymptotic one for M-estimators.
The exchange rate is the number of real samples $n_{\mathrm{eq}}$ that achieve the same risk with no synthetic data. It follows by setting $d_{\mathrm{eff}}/(2n_{\mathrm{eq}})$ equal to the right-hand side. Two consequences follow, and the experiments test both. First, the exchange rate is not a constant: it saturates once the bias term dominates. That is consistent with the observation that simple synthetic data can degrade tracking while richer synthetic data helps (On the Equivalency, Substitutability, and Flexibility of Synthetic Data). Second, the harm is directional. Only the component of $\Delta\eta$ that the Fisher metric weights heavily matters, and different downstream metrics are sensitive to different components.
The new part is not the general idea that a task-restricted discrepancy controls transfer. It is the specific claim that, in dual coordinates, data mixing is a straight line, so the substitution curve, the optimal mix and the per-metric damage all follow from one estimated vector $\Delta\eta$. For conditional models such as logistic regression, the dual-coordinate statement holds only to second order, not exactly. We state this up front and test whether the approximation is good enough.
Where this sits in prior work
The geometric ingredients come from the information-geometry literature. An Elementary Introduction to Information Geometry lays out dually flat manifolds, in which a convex potential and its Legendre conjugate give two coordinate systems. Straight lines in the conjugate ("mixture") coordinates are one of the two kinds of geodesic, and the Bregman divergence is the canonical distance. Generalized Legendre Transforms Have Roots in Information Geometry shows that generalized Legendre transforms reduce to the ordinary conjugate of an affinely deformed function. For us this means the choice of coordinates is a gauge choice that cannot change the predicted risk, which serves as a sanity check on the implementation. Geometric Structures Induced by Deformations of the Legendre Transform studies deformed transforms via link functions. It points to how the analysis might extend beyond canonical links, which we leave as future work. All three supply method; none of them considers data mixing or synthetic data.
The motivation comes from three synthetic-data papers. Downstream Task-Oriented Generative Model Selections on Synthetic Data Training for Fraud Detection Models finds that a neural tabular generator (CTGAN) distorts the class balance of credit-card fraud data. It keeps area under the ROC curve (AUROC) and recall but drives precision toward zero, while a Bayesian-network generator that preserves marginals wins on precision and accuracy.
Our framework gives a specific explanation. A class-prior shift moves $\Delta\eta$ mainly along the intercept direction of a logistic model. Ranking metrics such as AUROC are invariant to intercept shifts; threshold metrics such as precision are not. Structured Evaluation of Synthetic Tabular Data evaluates generators by asking whether $Q=P$ at increasing structural orders (marginals, pairs, leave-one-out conditionals, the full joint). That view implicitly treats every discrepancy as equally costly. The fraud result contradicts that view, and our Fisher-weighted projection is designed to resolve the contradiction. On the Equivalency, Substitutability, and Flexibility of Synthetic Data measures an exchange rate of roughly 30:1 synthetic to real frames for multi-person tracking, and notes that no general formulation exists. We supply one for likelihood-trained models, at the cost of not covering full end-to-end trackers.
Deep Legendre Transform learns convex conjugates by regression using the Fenchel identity. It could be used to evaluate $B_{F^*}$ when $F^*$ has no closed form, but it does not touch data evaluation. The statistical-correction line, AutoEval Done Right and Using Imperfect Synthetic Data in Downstream Inference Tasks, constrains the proposal. Those methods already debias estimates that use synthetic data. We must therefore show that our method answers a different question: how much synthetic data to train on, and which generator to use. It should not compete with them on confidence intervals.
Research Program
The core object to build is an estimator of $\Delta\eta$ and $G$ from a synthetic dataset and a small real sample. For canonical-link models there is a convenient identity. The real-data gradient of the loss, evaluated at the model fitted on synthetic data, equals $\eta_P - \eta_Q$ up to the covariate-shift correction. So $\Delta\eta$ costs one forward-and-backward pass over the real sample. $G$ is estimated from the empirical Fisher at that model.
There is one statistical trap, and the program treats it as a first-class problem. The plug-in quadratic form $\widehat{\Delta\eta}^\top \widehat G^{-1}\widehat{\Delta\eta}$ is biased upward by roughly $d/n_r$. With small real samples, this inflation can swamp the true bias. We will use a split-sample unbiased estimator and report it alongside the plug-in. This is the same small-sample regime where GMM-Synth's covariance weighting becomes unstable, so we expect to inherit some of its difficulty.
On top of the estimator we build three outputs. The first is a predicted substitution curve $\mathcal{E}(n_r, n_s)$ across all mixes. The second is a predicted optimal synthetic fraction and exchange rate. The third is a directional decomposition: we project $\Delta\eta$ onto parameter subspaces (intercept versus slopes, per-feature blocks) to predict which downstream metrics a given generator will damage.
For log-linear models of categorical tables, the dual-flat formula is exact. That makes the census table the cleanest testbed. Logistic regression is the main conditional case. As a stretch case, we apply the analysis to the final classification layer of a frozen network trained on synthetic data, treating the features as fixed.
Baselines
The naive reference is training on real data only at each $n_r$, together with the "always mix everything" policy that uses all available synthetic data. Any useful predictor must recommend mixes that beat both.
The strongest competitor for predicting the curve is the empirical substitution curve itself. This is the retraining protocol of On the Equivalency, Substitutability, and Flexibility of Synthetic Data, in which the downstream model is retrained at a grid of real fractions and each is tested on real data. We run this grid as ground truth with a large held-out real test set. We also run a budget-matched version: the same small real sample used by our method serves as the validation set, with the same number of downstream fits allowed. That is the fair comparison, because with unlimited real validation data, retraining wins by definition. That paper reports an exchange rate of roughly 30:1 and finds that 20% real data after synthetic pre-training nearly matches 100% real data, but on tracking, which we do not reproduce. Our method beats the budget-matched grid if, at equal real-sample budget and with one downstream fit instead of a grid, it chooses a mix whose true real-data risk is at least as low, more often across generators and seeds.
The second family of competitors is generic synthetic-data quality scores: the structural-tier scores of Structured Evaluation of Synthetic Tabular Data, computed with their released tiers, and an unweighted full-distribution divergence between $Q$ and $P$. These are compared on how well they rank generators by measured downstream risk. Specific numbers are not available here: no reported numeric result was found on Lacuna for those structural scores (the excerpt gives only qualitative orderings).
For inference tasks we compare against GMM-Synth and PPI++, using the published estimators. GMM-Synth reports that 100 human labels combined with synthetic data match 200 human labels alone, with MSE often reduced by over 50% and valid 95% coverage (Using Imperfect Synthetic Data in Downstream Inference Tasks). PPI++ reports effective-sample-size gains of 20% to 50% (AutoEval Done Right).
Two ablations isolate the key ingredient. One replaces $G^{-1}$ with the identity, giving a Euclidean gap in dual coordinates. The other replaces the task-projected gap with the full divergence. If either ablation predicts curves as well as the full method, the geometry is not earning its place.
Why not just…?
Why not just use PPI++ or GMM-Synth? These methods correct an estimate after the fact. They shrink toward the real-only answer when the synthetic source is uninformative, but they do not say which generator to use or how many synthetic rows to train a predictive model on. On the computational social science regression tasks there is real overlap, because the downstream target there is a parameter. A skeptic could argue that GMM-Synth's weighting already solves our problem. We will test this directly. We feed the mix our method recommends into GMM-Synth and compare its MSE with GMM-Synth on the default mix. Our method then serves as a front-end that chooses the data, not a replacement for the estimator. If the recommended mix gives no gain, we will report that our method adds nothing for inference and restrict the claim to training.
Why not just run Train-on-Synthetic-Test-on-Real on held-out real data? With ample real validation data, you should do this. The case for our method is the small-real-sample regime, where a held-out real loss estimated from a few hundred rows is too noisy to distinguish nearby mixes, and where each point on the curve needs its own retraining. The fraud results show a second problem with this approach. Its outputs fragment across metrics without explaining why, and the directional decomposition is meant to supply the reason. The budget-matched baseline above settles the question empirically. This is a genuine threat, and if it matches us at equal budget, the contribution reduces to the explanatory decomposition.
Why not just score generators with structural fidelity metrics? These metrics measure whether $Q$ matches $P$ without asking whether the downstream model cares about the mismatch. The fraud example shows this can rank generators in the wrong order for a given metric. The ablation that swaps in the full divergence tests whether task projection matters. If structural scores rank generators by downstream risk as well as our method, the simpler tool wins.
Proposed Experiments
The tabular core uses the SDV census, expedia and student tables from Structured Evaluation of Synthetic Tabular Data and the credit-card fraud data from Downstream Task-Oriented Generative Model Selections. The synthetic sources are synthpop, the cross-categorization model (PCC), CTGAN and a Bayesian-network synthesizer. Real-sample budgets range from 50 to 5,000 rows. Downstream models are log-linear models (where the formula is exact) and logistic regression (where it is approximate). For each pairing, the primary metric is the rank correlation and absolute error between the predicted and measured real-data risk across the mixing grid. We also measure the regret of the recommended mix relative to the oracle best mix.
The decisive directional check is on fraud. The prediction is that the intercept-direction component of $\widehat{\Delta\eta}$ predicts the precision drop across generators, while the slope-direction component predicts the AUROC drop. If a single undifferentiated gap predicts both equally well, the decomposition claim fails.
The inference arm uses the politeness, climate-stance and congressional-bills tasks from Using Imperfect Synthetic Data in Downstream Inference Tasks, reporting coefficient MSE and coverage.
A final check probes the limits of the method. We apply the final-layer version to a frozen image backbone with synthetic fine-tuning data and ask whether a saturating exchange rate appears, as the formula predicts.
The proposal is falsified if the Fisher-weighted prediction is no better than the Euclidean ablation, or if the budget-matched empirical grid consistently selects better mixes.
References
- On the Equivalency, Substitutability, and Flexibility of Synthetic Data
- An Elementary Introduction to Information Geometry
- Generalized Legendre Transforms Have Roots in Information Geometry
- Geometric Structures Induced by Deformations of the Legendre Transform
- Downstream Task-Oriented Generative Model Selections on Synthetic Data Training for Fraud Detection Models
- Structured Evaluation of Synthetic Tabular Data
- Deep Legendre Transform
- AutoEval Done Right: Using Synthetic Data for Model Evaluation
- Valid Inference with Imperfect Synthetic Data