Do Synthetic Markets Fake Equilibrium Stability? Auditing the Concavity Margin of Data-Induced Games

When researchers test learning algorithms for competing agents, such as pricing bots, spam filters facing adaptive spammers, or energy aggregators, they almost always test them on synthetic data. The theory behind these algorithms depends on a geometric property: the game the agents play must be "diagonally strictly concave." In plain terms, the combined update directions of all players must point inward toward a single equilibrium, like water draining into one funnel. When that holds, there is exactly one stable outcome and simple gradient-following learners reach it. This note argues that synthetic data generators can make this funnel look steeper than it really is. Generators that fit a single smooth population model average away the polarized, multimodal behavior that real populations show. Averaged behavior yields smooth, concave aggregate responses. The game built from synthetic data therefore satisfies the uniqueness condition with room to spare, while the game built from real data may satisfy it barely or not at all. If so, synthetic validation systematically overstates how stable and convergent multi-agent learners are.

We propose to measure this effect directly. For any dataset, real or synthetic, we build the game it induces and compute its concavity margin: how strongly the players' joint gradient field contracts toward equilibrium. We then evaluate a synthetic generator by how well it preserves that margin and the equilibrium's location, not by how closely its samples match real samples statistically. The central claim is testable. Across a family of generators, the gap in concavity margin between synthetic and real data should predict how much a multi-agent learner degrades when moved from synthetic to real data, and it should predict this better than distributional distances do.

Core Claim

Let $F(x)$ be the pseudo-gradient of a game: the stacked vector of each player's payoff gradient with respect to their own action, evaluated at the joint action $x$ in feasible set $X$. Let $J(x)$ be its Jacobian. The quantity we audit is the monotonicity margin

$$\mu(G) = \min_{x \in X} \lambda_{\min}\!\left(-\tfrac{1}{2}\big(J(x) + J(x)^\top\big)\right),$$

where $\lambda_{\min}$ is the smallest eigenvalue. A positive $\mu$ is the classical diagonal-strict-concavity condition. It guarantees a unique Nash equilibrium, meaning a joint action from which no player gains by deviating alone, and it makes projected gradient play contract at a rate that grows with $\mu$. That rate is the standard strongly-monotone contraction argument. The generalized version of Altman, Hanawal and Sundaresan allows player-specific weightings and a wider class of games, so we compute $\mu$ under the best weighting found by a small outer search. Writing $G_{\text{syn}}$ and $G_{\text{real}}$ for the games induced by synthetic and real data, the claim has two parts. The first is a direction: generators that collapse population heterogeneity produce $\mu(G_{\text{syn}}) > \mu(G_{\text{real}})$. The second is a predictive law: transfer degradation tracks the margin gap $\mu(G_{\text{syn}}) - \mu(G_{\text{real}})$ together with the equilibrium displacement $\|x^*_{\text{syn}} - x^*_{\text{real}}\|$.

The mechanism comes from two findings in the file that have never been put together. Learning Real-Life Approval Elections shows on Pabulib participatory-budgeting data that a single-parameter generator fits the mean of a polarized electorate and produces voters who resemble nobody, and that four-component mixtures are needed. Bracale et al.'s s-concave pricing work proves equilibrium uniqueness exactly when aggregate demand is $s$-concave, a family of curvature conditions in which $s=1$ is ordinary concavity and $s=0$ is log-concavity. These two facts connect through a basic property: a mixture of log-concave functions need not be log-concave. A market made of distinct customer segments can therefore break the shape condition that the uniqueness proof relies on, while any single-segment synthetic fit satisfies it automatically.

Relation to Prior Work

Three papers motivate the gap. In each, the uniqueness geometry is imposed on the synthetic generator itself, so it cannot fail in evaluation. SPE-BR for s-concave demand is validated only in simulated 2-, 4- and 6-seller markets whose demand is $s$-concave by construction. PML-GLUCB under generalized linear demand assumes a known monotone link function and is tested in matching simulations. Online performative gradient descent is tested on idealized linear and kernel-based data maps. None of them asks whether the data its generator produces would satisfy the condition if the generator were replaced by reality. Our audit asks exactly that.

Two papers supply the method. Leon et al.'s sum-of-squares hierarchy certifies monotonicity of polynomial games through a sequence of semidefinite programs and can project an ill-behaved game onto the nearest certifiably monotone one. We reuse it to obtain certified lower bounds on $\mu$ after fitting polynomial surrogates to data-induced games. This extends their work from random-coefficient polynomials to games estimated from data, and we report the fitting error that this extension introduces. Ratliff, Burden and Sastry's local characterization gives the block-Hessian conditions we use when $\mu$ is negative, to tell whether the real game has several isolated local equilibria or none that are well behaved.

Two papers shape the evaluation design. Rosenberg-Kima and Pardos compare simulated and real data by the shape of the likelihood landscape over model parameters rather than by predictive accuracy. Our audit follows the same logic but replaces the single-agent likelihood landscape with the multi-agent equilibrium landscape. Chang et al. define substitutability and an exchange rate of roughly 30 synthetic frames per real frame for tracking. They also show that behavioral complexity, not visual realism, drives utility, raising DanceTrack HOTA from 68.8 to 73.5. We borrow their substitutability protocol for strategic data, where the hypothesis is that the margin gap sets the exchange rate.

One paper partly contradicts our setup. Brückner and Scheffer find that Nash-based spam filters beat minimax filters on real chronological email. Their result supports targeting equilibria, but it also shows that their uniqueness condition held well enough on real data. We treat it as a case where we predict a small margin gap, and we test whether that prediction holds.

Research Program

The program builds a margin-audit tool and applies it to two testbeds.

The tool takes a dataset and a game template, meaning the players' loss or payoff forms with the data as a parameter. It returns three things. The first is a sampled estimate of $\mu$, taken as the minimum over a dense grid and adversarial search of $x$. The second is a sum-of-squares certified lower bound on $\mu$ for a polynomial surrogate, reported with the surrogate's fitting error. The third is the equilibrium $x^*$, or the set of local equilibria when the margin is negative.

The first testbed is the static prediction game of Brückner and Scheffer on the public (65,000) and private (40,000) chronological email corpora. In this game, a learner's regularized loss and an adversary's cost-penalized distortion depend on the emails. The "real" game uses a later time period. The "synthetic" games use stand-ins for that future period: adversarial emails generated from the earlier period under the assumed adversary cost, and tabular generators of increasing flexibility fit to the earlier period.

The second testbed is pricing, and it is semi-synthetic because the file contains no real retail transaction data. We keep the multi-seller simulator protocol of Bracale et al. but replace its reference market with mixtures of $s$-concave customer segments. The segment weights and the spread across segments are calibrated from the multimodality of the fitted four-component approval mixtures on 271 Pabulib elections. The synthetic generators are the deployed algorithms' own assumptions, a single-index $s$-concave fit and a generalized-linear fit, plus mixture fits with 2, 3 and 4 components. We are explicit that this testbed checks the mechanism under realistic heterogeneity and does not establish external validity for real retail markets.

On both testbeds the downstream quantities are the same. For pricing, we pre-train or validate SPE-BR and PML-GLUCB on each generator's data, deploy them in the reference environment, and record three outcomes: the Nash gap after $T$ rounds, the time to reach a fixed Nash-gap tolerance, and how often trajectories cycle or settle on non-competitive (collusive-looking) prices. For the spam game, we record future-period AUC. We also measure strategic substitutability. Following Chang et al., this is the fraction of real exploration periods, or real training emails, that synthetic data can replace before the final Nash gap or AUC falls below the result from real data only.

Baselines

The naive reference point is the current practice: validate on synthetic data and report the result as the expected deployment result. This is how all three pricing and performative papers are evaluated. Beating it requires showing a measurable optimism bias, meaning synthetic-data Nash gaps and convergence times that are reliably better than those observed in the reference environment.

The strongest competing predictor of transfer is a distributional-distance audit. This includes the divergence-based validation of Synthetic Tabular Data Validation: A Divergence-Based Approach and kernel mean-embedding distance, computed between each generator's samples and the real sample. It is strongest because it is cheap, needs no game model, and is what practitioners already use. We give it the same real-sample budget as our audit and tune its kernel bandwidth or divergence choice on the same held-out generators. No reported result was found on Lacuna for divergence-based audits predicting equilibrium transfer in any of our testbeds, so the comparison has to be run fresh. Our audit beats it if its rank correlation with observed transfer degradation across generators is higher with non-overlapping bootstrap intervals. The second competitor is a Rosenberg-Kima-and-Pardos-style comparison of single-agent likelihood landscapes, which tests whether multi-agent structure adds anything. For the spam game, the strongest method being evaluated is the Nash filter itself. Brückner and Scheffer report that it beats SVMs, logistic regression and minimax "by several standard errors" but give values only in figures, so no reported numeric result was found on Lacuna. We therefore reproduce their pipeline as the anchor.

Two ablations isolate the key ingredient. One uses equilibrium displacement alone without the margin, and the other uses the margin alone without displacement. A third replaces the certified sum-of-squares bound with the sampled estimate, which shows whether certification matters in practice or only makes the guarantee formal.

Why not just…?

Why not just test the learner on held-out real data? When abundant real interaction data exists, this is the right answer and our audit adds little. The audit is meant for the case that motivates synthetic data in the first place, where real strategic data is scarce. In that regime a full deployment run is noisy or impossible, but $\mu$ depends on local second-order structure that may be estimable from far fewer samples. This is a real threat, so we test it directly. We subsample the real data and compare how fast the audit's prediction and a direct small-sample transfer test each converge to the full-data transfer outcome. If the direct test is as sample-efficient, the audit's practical case collapses to a diagnostic explanation, and we will report that.

Why not just use statistical distance between synthetic and real samples? Chang et al. show that surface similarity and downstream utility come apart, and Li et al. (federated renewable scenario generation; Privacy-Preserving Spatiotemporal Scenario Generation of Renewable Energies: A Federated Deep Generative Learning Approach) report statistically similar scenarios that still fail operationally. Neither result involves equilibria, however, so the head-to-head rank-correlation experiment above decides the question.

Why not just fit mixture generators and stop worrying? The Pabulib study already shows that mixtures fit real electorates better. A mixture generator is a fix, though, not a test. It cannot say how many components preserve the equilibrium structure, and better likelihood does not guarantee the margin is preserved. Mixture fits are included as treatments, and the audit should identify the component count at which the margin gap closes.

Why not just train robust minimax policies that ignore equilibrium structure? Brückner and Scheffer find minimax too pessimistic when the adversary model is roughly right. Minimax policies are included as a deployment comparison in the spam testbed for generators with large margin gaps, where our account predicts they should regain their advantage.

Decisive Checks

The proposal stands or falls on three results. The first is direction: in the pricing testbed, single-index and generalized-linear generators must show $\mu(G_{\text{syn}}) > \mu(G_{\text{real}})$ on most Pabulib-calibrated markets, and the gap must shrink as mixture components are added. The second is prediction: across all generators and both testbeds, the margin gap plus displacement must rank transfer degradation better than the distributional-distance baseline, in a pre-registered rank-correlation comparison. The third is consistency with the anchor case: in the spam game, where Brückner and Scheffer's Nash filter worked on real data, the margin gap must be small. A large gap there with good transfer would falsify the mechanism. If the first check fails, synthetic data is not faking stability in these settings, and that negative result is worth publishing because it would support the current validation practice in this literature.