When a model is trained on synthetic data, two knobs can be turned. The first is how much data to generate. Call this reach. The second is how rich each sample is: how many interacting agents, occlusions and crossing paths a simulated scene contains. Call this range. Economists who study platforms and firms have documented a tradeoff between these two. Simple, standardized transactions scale to huge networks, while deeply coordinated ones only work at small scale, and pushing scale past a threshold makes coordination costs dominate. We propose to test whether synthetic training data obeys the same kind of production law. The question is concrete: under a fixed budget of simulation compute, is downstream accuracy maximized by many cheap simple scenes, by few expensive complex ones, or by an interior mix whose location can be predicted from the complexity of the real target data?
The practical stake is the "exchange rate" between synthetic and real data. On the Equivalency, Substitutability, and Flexibility of Synthetic Data measured that about 30 synthetic frames are worth one real frame for multi-person tracking. Their study treats this number as if it were a property of the generator. We claim it is not a constant. It should depend on how closely the behavioral complexity of the synthetic scenes matches that of the target data, and volume and complexity should act as complements rather than substitutes. Adding more simple data cannot buy what complexity provides, which is why simple synthetic data actually hurt on a hard tracking benchmark in that paper. If this holds, a practitioner could fit a small production model from a pilot sweep and choose a budget allocation, instead of guessing.
Core Claim
Let $V$ be the number of synthetic frames and $C$ a measured complexity level of the scenes. For tracking, $C$ combines the number of agents, the number of groups, occlusion rate and trajectory-crossing rate. Let $c(C)$ be the measured compute cost per frame at that complexity and $B$ the total generation budget. We model downstream tracking accuracy after fine-tuning on a fixed real fraction as a production function $P(V, C)$ and pose the allocation problem
$$\max_{V, C} \; P(V, C) \quad \text{subject to} \quad V \cdot c(C) \le B.$$
The specific, falsifiable claims are as follows. First, $P$ is best described by a form in which the two inputs interact. In the vocabulary of The Size Conundrum: Why Online Knowledge Markets Can Fail at Scale, an "interactive essential" form should fit better than a separable or "substitutable" form. Second, because $c(C)$ grows with complexity, the constrained optimum lies at an interior complexity. Third, that optimal complexity tracks the complexity statistics of the real target set, so it is higher for DanceTrack than for MOT17. Fourth, the equivalency ratio rises as synthetic complexity falls below target complexity.
Prior Work and Its Role
The synthetic-data papers supply both the evidence and the testbed. On the Equivalency, Substitutability, and Flexibility of Synthetic Data trains the MOTRv2 tracker on the procedural Unity generator M³Act. It finds that complex multi-group scenes raise DanceTrack HOTA (Higher Order Tracking Accuracy) from 68.8 to 73.5, while single-walker scenes degrade it. That study motivates us. However, it treats complexity as three or four categorical presets and never prices them. We turn the categories into a continuous, cost-aware axis and fit a frontier over it.
Improving the Scaling Laws of Synthetic Data with Deliberate Practice steers a diffusion generator toward samples the classifier is uncertain about. It reaches 50.2 ImageNet-1k top-1 accuracy with 1.3M samples, versus 45.8 with 13M static samples. This is the strongest evidence that which samples matter more than how many, and it supplies our strongest competitor. Its notion of difficulty is per-image classifier entropy, not the structured interaction complexity of a scene, and it gives no cost model.
BeyondWeb shows the same lesson at trillion-token scale: restructured rephrasing beats raw volume, and uniform synthetic scaling plateaus. It constrains us by warning that a single complexity axis may eventually saturate. Its remedy is heuristic diversity rather than a fitted tradeoff.
The economics papers supply method and functional form, not evidence about neural networks. The Size Conundrum fits Cobb-Douglas-style production functions under four interaction types (essential, interactive essential, antagonistic, substitutable). It selects among them with RMSE, explained variance and AIC (the Akaike Information Criterion, a fit score that penalizes extra parameters). We reuse that model-selection procedure directly. Can Digital Transformation Definitely Improve Firms' Markups? uses threshold regression to locate an inverted-U turning point, where more digitization starts to lower returns. We borrow the threshold-estimation idea to locate the complexity past which extra cost per frame stops paying off. Business Strategy in E-bussiness environment provides the reach-versus-range framing itself.
We are explicit that these are analogies. The proposal stands or falls on the tracking experiments, not on the economic parallel.
We deliberately leave aside the recursive-collapse literature (The Curse of Recursion, Is Model Collapse Inevitable?). Our generator is a procedural simulator, not a model retrained on its own outputs, so collapse is not the mechanism at stake here.
Research Program
We will first make complexity measurable and priced. M³Act exposes scene parameters: agent count, group structure, motion type and camera layout. We will define $C$ from properties computed on the rendered output, namely mean occlusion fraction, trajectory crossings per agent-second, and agents per frame. The same statistics are computed on MOT17 and DanceTrack training annotations to give target complexities $C^*_{\text{MOT17}}$ and $C^*_{\text{Dance}}$. Cost $c(C)$ is measured in wall-clock GPU-seconds per rendered and annotated frame, under one fixed hardware setup.
Next we run a factorial sweep. The grid has roughly five complexity levels and five synthetic volumes, spanning the range in which Chang et al. operated. Each cell is pretrained with MOTRv2 under a fixed number of optimizer steps, then fine-tuned on 20%, 40% and 100% of the real training set. Three seeds per cell give an estimate of noise.
We fit the four interaction forms from Dev et al. to these results and compare them by AIC. We also fit a threshold regression in $C$. The fit uses only a pilot subset of cells, chosen as low and mid volumes. Its predictions are then scored on the held-out high-volume cells and on the budget-optimal allocations it recommends. Finally, the equivalency ratio is computed per complexity level as the number of synthetic frames needed to match the gain from one additional real frame, and regressed on the gap $C - C^*$.
Baselines
The naive reference point is training MOTRv2 on real data alone at each real fraction, with no synthetic pretraining. Every synthetic configuration must beat this to matter. The second reference is volume-maximizing allocation: spend the whole budget on the cheapest, simplest scenes. This is the "reach" strategy, and Chang et al. report it can fall below the real-only baseline on DanceTrack.
The strongest fixed-preset baseline is the multi-group configuration of Chang et al., reported at 73.5 HOTA on DanceTrack against 68.8 without it. The file gives no MOT17 HOTA value; Chang et al. report only that synthetic pretraining with 40% real data exceeds 100% real data alone. We will regenerate this preset ourselves at the matched budget rather than trusting the absolute numbers, since our hardware and training steps will differ.
The strongest adaptive competitor is an uncertainty-guided generator in the spirit of Deliberate Practice, transplanted to scenes. At each stage it samples scene parameters where the current tracker's association uncertainty is highest. Like Deliberate Practice, it uses a small real validation set for patience-based stopping. We cannot import Askari-Hemmat et al.'s ImageNet numbers to tracking; no reported result was found on Lacuna for uncertainty-guided generation on MOT17 or DanceTrack.
All methods share the same simulation budget $B$ in GPU-seconds, the same pretraining and fine-tuning steps, the same real subsets, and the same hyperparameter search of eight trials on a real validation split. The uncertainty-guided method also has its own selection cost charged against $B$.
Two ablations isolate our key ingredient. The first fits a separable model in which each complexity level gets its own power law in volume, with no interaction term. The second uses the frontier allocation while ignoring $c(C)$, treating all frames as equally expensive.
Beating these baselines means the frontier-recommended allocation matches or exceeds the multi-group preset and the uncertainty-guided generator in HOTA at equal budget, at all three budgets, by more than the seed standard deviation. Alternatively, it means matching them at measurably lower budget.
Why not just…?
Why not just use active, uncertainty-guided generation and let it discover the right complexity? This is the real threat. Deliberate Practice already shows that targeting hard samples beats raw scale by an order of magnitude (Improving the Scaling Laws of Synthetic Data with Deliberate Practice). Our proposal differs in three ways. It is cost-aware. It is predictive before any data is generated. And it does not need a real validation anchor during generation, a dependence that Deliberate Practice itself notes. If the uncertainty-guided baseline matches the frontier allocation at equal budget, we will report that the production model is descriptive rather than prescriptive. Its remaining value would then be the exchange-rate analysis alone.
Why not just grid-search complexity and volume directly? Grid search is what produces the data we fit, so it is not an alternative to measurement. Rather, it is the cost we aim to avoid at deployment. The test is whether a fit on pilot cells predicts held-out high-volume and budget-optimal cells. If separable per-level power laws predict as well as the interaction form, the complementarity claim is false, and the simpler recipe wins.
Why not just always generate the most complex scenes available? Chang et al. show complexity helps on DanceTrack. However, the most complex scenes are also the most expensive to simulate, and MOT17's target complexity is lower. Our sweep directly compares max-complexity allocation against the interior optimum at equal budget on both benchmarks. If max-complexity wins everywhere, the inverted-U claim fails.
Decisive Checks
The idea survives only if four checks hold. First, an interaction form must beat the separable form by AIC on held-out cells. Second, the budget-constrained optimum must sit at interior complexity on at least one benchmark. Third, the optimal complexity for DanceTrack must exceed that for MOT17, in the direction predicted by the measured $C^*$ values. Fourth, the equivalency ratio must vary systematically with $C - C^*$ rather than hovering near the single 30:1 figure.
A secondary replication swaps in the procedural tumor generator of Synthetic Data as Validation, with tumor shape irregularity and mass effect as the complexity axis. The baseline there is the reported improvement in Dice score (overlap with the true segmentation) from 31.1% to 35.4% on FLARE'23. This replication tells us whether the frontier is specific to tracking or a general property of procedural synthetic data. A failure on any of the first three checks in tracking would be reported as a negative result. Such a result still matters, because it would mean synthetic exchange rates cannot be priced by scene complexity.
References
- On the Equivalency, Substitutability, and Flexibility of Synthetic Data
- The Size Conundrum: Why Online Knowledge Markets Can Fail at Scale
- Improving the Scaling Laws of Synthetic Data with Deliberate Practice
- BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining
- Can Digital Transformation Definitely Improve Firms’ Markups?
- Business Strategy in E-bussiness environment
- The Curse of Recursion: Training on Generated Data Makes Models Forget
- Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
- Synthetic Data as Validation