Do LLM Agents Respond to the Externality or to the Story?

Simulations in which language-model agents set prices or harvest a shared resource now report strong findings. Agents collude, over-extract, and can be reined in by governance rules written in plain English. Almost every one of these findings comes from a hand-written toy environment: a logit demand curve, or a stylized catastrophe-risk equation. Almost none checks whether the agents' choices actually move when the economic coupling between agents changes. Economists call that coupling an externality: one agent's action changes another agent's payoff. This note proposes a dose-response test. We turn the strength of the externality up and down, anchor one point on that dial to coupling estimated from real retail transactions, and measure whether agent behavior follows the dial the way a best-responding agent's would. Two outcomes are possible. If behavior tracks the dial, the externality is doing the work. If behavior stays flat while the narrative framing moves it, the reported collusion and governance effects are artifacts of storytelling and of the synthetic environment.

The opportunity is that two known weaknesses of this literature can be attacked by one experiment. The first is reliance on synthetic environments, which may build in the very dynamics being "discovered." The second is the identification problem: an agent's output may reflect training-data retrieval or prompt compliance rather than strategic interaction. A dose-response curve on an empirically anchored externality addresses both. Retrieval and role-play have no reason to follow a numeric coupling parameter the agent can only infer from market feedback. The empirical anchor tells us whether the synthetic environment sat anywhere near the real regime.

Core Claim

A finding from an LLM externality simulation should count as evidence about strategic behavior only if it passes two tests. First, agent actions must respond to the externality strength with roughly the slope that best-response reasoning predicts. Second, the finding must survive replacing the synthetic demand model with one estimated from real transaction data. We define an externality-tracking ratio

$$S = \frac{\partial \bar a / \partial \rho}{\partial a^{BR} / \partial \rho},$$

where $\rho$ is the externality strength (here, the correlation between consumers' valuations of goods sold by different agents), $\bar a$ is the agents' mean action (price markup, or extraction level), and $a^{BR}$ is the analytically computed best-response or equilibrium action at that $\rho$. A value of $S \approx 1$ means agents track the coupling as a strategic agent would. A value of $S \approx 0$ means their behavior is set by something other than the payoff structure.

We predict that current LLM agents will show substantially attenuated $S$ under familiar economic framing. We also predict that relabeling goods with nonsense names will change behavior more than changing $\rho$ does. If that holds, it directly undermines how existing governance and collusion results are interpreted.

Grounding and Prior Work

The seed studies motivate the question. Distinguishing Governance Effects from Prompting Artifacts runs three LLM retailers for 8 rounds in a synthetic market. Against a profit-only baseline and a managerial-tone placebo, a governance prompt lowers violation rate (0.281 to 0.219) and coordination-follow rate (0.250 to 0.156) with mean price unchanged. Its placebo controls for prompt tone, but its demand is entirely synthetic and the externality strength is never varied. Individually Sensible, Collectively Harmful finds that 396 of 400 commons episodes end with negative welfare. It shows that behavior sits in a low-regret band that simple heuristics also reach, and that prompt wording is a first-order variable. That last point is precisely why we need a non-linguistic lever.

Pricing work points the same way. Algorithmic Collusion by Large Language Models shows duopoly LLM agents pricing well above competitive levels. In that study, injecting a "price-war" thought raises prices by about 5% of the monopoly markup. Prompt Optimization Enables Stable Algorithmic Collusion shows that optimizing prompts for profit alone yields stable tacit collusion under nested-logit demand. Both use closed-form demand, and neither asks whether behavior scales with the coupling between firms.

One paper constrains the design. Position: LLM Social-Simulation Agents… Without an Identification Strategy argues that training retrieval, prompt compliance, and genuine interaction are observationally equivalent in simulation logs. It proposes a checklist built from ablations, placebos, and leakage audits. We adopt its relabeling and placebo ideas. We add what it leaves open: a quantitative identification lever for dynamic games, namely the slope of behavior with respect to a payoff parameter.

Another paper supplies the method. Latent Variable Copula Inference for Bundle Pricing estimates correlated consumer valuations from the Ta-Feng grocery transactions, roughly 120,000 baskets. It uses a Gaussian copula, a way to glue separately estimated per-item demand curves together with a correlation structure. The paper shows that assuming independence distorts profit curves enough to destroy revenue. That estimator is static and single-seller, and its cost grows exponentially in bundle size. Our 2–3 agent markets fit inside that limit.

MarketSim is the closest grounding effort. It reaches a 3.48% price-tracking error by feeding real news to LLM traders. Its grounding lives in the agents' information, not in the payoff coupling between firms, and it does not study product-market externalities.

Research Program

We build a pricing environment with 2 or 3 sellers. Each seller prices one item from a Ta-Feng item pair or triple, chosen to span empirically estimated valuation correlations: strong substitutes, near-independent items, and complements. Consumer demand comes from the copula model fitted to Ta-Feng, so the externality one seller's price imposes on another is set by real basket data rather than an assumed logit. The externality dial $\rho$ is the copula correlation. We sweep it from 0 to beyond the empirical estimate, holding the per-item demand curves fixed so that only the coupling changes. This also yields analytic best-response and joint-profit-maximizing prices at every $\rho$, which supply $a^{BR}$ and the collusion reference.

Each market is crossed with three further factors. The first is framing: real grocery category names versus nonsense commodity labels with identical numbers. The second is instruction: profit baseline, Tulabandhula-style placebo, and governance policy. The third is environment provenance: Ta-Feng copula demand versus a synthetic logit demand matched to the same own-price elasticities and the same induced cross-price elasticity at each $\rho$. Agents observe only prices, quantities, and their own profits, so $\rho$ must be inferred from feedback.

For the commons side, we apply the same dial to Creeping Trap's externality parameters: the slope of catastrophe risk and of stakeholder harm in aggregate extraction. No real harvest or common-pool dataset was found in the research file, so the commons arm tests only the dose-response and relabeling parts and stays synthetic by necessity.

Baselines

The naive reference point is the seed protocol as published. That means synthetic demand, a single fixed coupling, and familiar labels, run with DeepSeek-V4-Flash and the three instruction conditions of Distinguishing Governance Effects. The comparison numbers are those reported there: violation rate 0.281±0.121 versus 0.219±0.152, coordination-follow 0.250±0.131 versus 0.156±0.108, and mean price 2.887 versus 2.881. We rerun this protocol with more seeds (40 per cell rather than 12) so that changes in the governance effect can be resolved.

The strongest methodological competitor is the identification checklist of Lin & Liu, applied alone: prompt ablations, the placebo, and nonsense relabeling, all at fixed $\rho$. It is strong because it already targets retrieval and role compliance. Our protocol beats it if $S$ separates conditions or models that the checklist rates as indistinguishable. A concrete case is a model whose behavior is stable under relabeling but still flat in $\rho$.

The strongest agent baseline is a myopic best-response heuristic and a gradient-following price-adjustment rule. These have the same feedback and the same horizon as the LLMs. They are strong because Individually Sensible, Collectively Harmful shows such heuristics reach the same low-regret region as LLMs. They also calibrate $S$: a correctly implemented best responder should have $S \approx 1$, which validates the measurement.

For collusion levels, the comparison point is the implantation effect in Algorithmic Collusion by Large Language Models: about 5% of the monopoly markup. We have no reported Q-learning numbers to compare against; no reported result was found on Lacuna for Q-learning pricing agents in these environments.

The ablation that isolates our key ingredient is the matched-elasticity logit arm. It answers whether real-data coupling matters beyond matching first-order elasticities.

All arms share models, seeds, round counts, and a fixed token budget per episode. Prompts are frozen before the sweep, with no per-condition tuning.

Why not just…?

Why not just calibrate the logit demand to Ta-Feng elasticities and stop there? This is a real threat. If a logit matched to own- and cross-price elasticities reproduces every behavioral finding of the copula environment, then simple calibration suffices and the copula adds nothing. We run exactly this arm. If it matches within seed variance, we report that as the result: a cheap and useful fix for the field. Letham et al. give reason to expect divergence, because independence-style approximations distorted profit curves. That paper concerns bundle pricing by one seller, though, not strategic responses, so the question remains open.

Why not just run the Lin & Liu checklist? Relabeling and placebos can show that behavior is not tied to particular words. They cannot show that behavior is tied to the payoff structure. An agent could be label-invariant and still ignore the externality entirely, for example by charging a fixed "reasonable" markup. The dose-response slope tests the positive claim, and the baseline comparison above shows whether it separates cases the checklist does not.

Why not just use heuristic or bandit agents, which are cheap and transparent? For measuring the market itself, heuristics are better. The scientific claims at stake, however, concern how LLM agents respond to natural-language governance. Heuristics cannot read a policy, which is why Tulabandhula uses LLMs. In our design heuristics serve as the $S \approx 1$ reference, not as the object of study.

Why not just ground agents in real context, as MarketSim does? Grounding what agents read improves realism of their information, but it leaves the payoff coupling synthetic. It also makes retrieval more likely, not less. Our grounding is placed where the externality lives, in demand, and the agent never sees it named.

Proposed Experiments and Decisive Checks

The primary experiment estimates $S$ for each model (DeepSeek-V4-Flash, GPT-4o, Llama-3.3-70B, and a reasoning model such as o3-mini, motivated by the free-riding results in Corrupted by Reasoning) under each framing. Each estimate comes from a regression of final-round markup on $\rho$ across at least seven dial settings, with bootstrap intervals over seeds.

Secondary metrics are three. The first is markup as a fraction of the gap between competitive and joint-profit prices. The second is the governance effect relative to placebo, computed separately under copula and logit demand. The third is consumer and total welfare, computed from the copula model.

Three results would be decisive. If LLM agents show $S$ near zero while the heuristic shows $S$ near one, the field's pricing results are not responses to the externality. That holds regardless of how large the collusion or governance effects are. If the effect of nonsense relabeling on markup exceeds the effect of moving $\rho$ from zero to its empirical Ta-Feng value, framing dominates the economics. If the governance-minus-placebo effect changes sign or loses more than half its size when switching from the synthetic logit to Ta-Feng copula demand, then the synthetic environment was producing the governance finding. A clean negative on all three, meaning $S$ near one, relabeling robustness, and provenance robustness, would be equally valuable: it would give the first evidence that these simulations measure strategic response to coupling rather than narrative.

Workshop authors to connect with