When two AI agents bargain, share a resource, or split work, each would often cooperate if it could trust the other. Game theory has a precise result here. If players can see each other's intentions or source code before acting, mutual cooperation can become a stable, self-interested outcome even in the Prisoner's Dilemma. LLM agents are increasingly built as small teams, with a planner that decides what to do and an executor that carries it out on concrete data. That structure gives a natural way to make agents partly see-through. An agent can publish its planner's output, which is a short conditional policy, while its executor stays private, holding budgets, keys, and valuations. A neutral auditor then spot-checks whether the executor actually followed the published policy. The audit catches a mismatch with some probability $p$.
The core claim is that this probability $p$ plays the same role as the "leak probability" in the theory of translucent players. Cooperation should therefore switch on at a threshold that can be computed from the payoffs. Once past that threshold, publishing plans should recover most of the cooperation benefit of full transparency. It should cost far fewer tokens than sharing full reasoning traces, and it should leak much less private information. If this holds, a design question becomes an engineering rule. The rule says which sub-agent to expose, which to keep opaque, and how much auditing is enough.
Core Claim
The proposal rests on one structural observation. In a decomposed agent, the planner's output is literally an intended strategy and the executor's action is the actual strategy. Bayesian games with intentions introduced exactly this split to model preferences that depend on what others intend. That paper defines an equilibrium as a state in which intended and actual strategies coincide. Its treatment is purely abstract: intentions are mathematical objects, not computations. In a decomposed LLM agent, the intention is an inspectable artifact, and the gap between intention and action is something an auditor can measure.
Game Theory with Translucent Players shows that if a defection decision might "leak" before the opponent moves, cooperation can be rational. We make the leak concrete. The leak is the auditor detecting that the executor departed from the published plan. Detection happens before the opponent's action is committed, and the opponent then retaliates.
In a symmetric two-player game, the payoff labels are as follows. $T$ is the temptation payoff for defecting on a cooperator, $R$ is the reward for mutual cooperation, and $P$ is the punishment for mutual defection. A player who publishes a cooperative plan but secretly defects earns $P$ with probability $p$ and $T$ otherwise. Cooperation is therefore stable when
$$p \;\ge\; p^* = \frac{T-R}{T-P},$$
which follows directly from comparing the reward $R$ with the expected deviation payoff $pP + (1-p)T$.
This threshold is the analogue of the coupling threshold $\gamma^*$ that Parametric Open Source Games proves for gradient-trained players. Our version is stated in terms of a quantity a system designer controls: the auditing rate.
The second half of the claim concerns privacy. Game Theory with Simulation of Other Players proves that letting one player simulate another can backfire. Exact simulation turns the game into one where the simulated player leads, and in games that depend on secrecy, simulation can destroy good equilibria. Decomposition offers a way around this. Suppose the planner is built so that it sees only public state and writes a policy over coarse private categories. Then publishing the planner reveals the rule without revealing the inputs, so the agent is open about its policy and closed about its private data.
Relation to Prior Work
Three papers supply the theory and constrain the design.
Translucent players and Bayesian games with intentions provide the solution concepts. Both are stated for abstract, monolithic, one-shot players. We instantiate them in executable agents and treat the leak probability as an engineered audit rate rather than a psychological belief.
Proving Your Way to Cooperation formalizes agents that read each other's code in Lean. It also shows that "defect unless proof of cooperation" (DupocBot) is the bot that sustains fully cooperative equilibria. That work uses a two-agent LLM pipeline, one agent writing bots and another writing proofs, but only as an engineering tool. The game players themselves are one-line programs. We borrow its bot library as the vocabulary in which planners write their published policies, so that our policies are open-source programs in its sense. The difference is that our policies are executed on private data by a separate, opaque executor.
Parametric Open Source Games assumes noise-free access to the opponent's full parameter vector in 2×2 games. That paper motivates the threshold form of our prediction. It also warns that cooperation does not arise from a "cold start" without some built-in sensitivity to the opponent.
On the decomposition side, two systems supply the planner/executor pattern that we reuse unchanged. MACM separates a Thinker, a Judge, and an Executor. Self-collaboration Code Generation via ChatGPT separates an Analyst, a Coder, and a Tester. Neither system places agents in strategic opposition.
Two further works constrain the design. Detect Before You Attribute shows that LLM judges degrade on raw multi-agent traces and do best when shown only roughly 20–25% of a trace. This is direct evidence against "just share everything." ALYMPICS supplies an opaque multi-agent auction environment that we extend. That paper also found that persona prompts had no significant effect on outcomes, which warns that role labels alone do not change behavior.
Research Program
Each player is a two-module LLM agent.
The planner sees the game rules, public history, and the opponent's published policy. It writes a short policy in a restricted language. That language consists of conditionals over public observations, over the opponent's published policy, and over coarse private categories such as "my remaining need is high." This is the agent's intended strategy, and it is published.
The executor receives the policy plus the private state and emits the concrete action, such as a bid amount, a move, or an allocation. Executors may be honest, meaning prompted to follow the plan, or adversarial, meaning instructed to maximize payoff and free to depart from it.
A trusted auditor has access to private state. With probability $p$ it re-evaluates the published policy on that state and compares the result to the executor's action. A mismatch is announced before the opponent's action locks. The auditor reveals only pass or fail, never the private state itself.
The program has three analytical pieces.
The first piece is theory. We extend the threshold result from one-shot 2×2 games to repeated games with private types. We also add a term for honest executor drift: an LLM executor that fails to follow its own plan without any strategic intent. This drift creates false alarms that lower the effective punishment, so it should raise the threshold.
The second piece measures how often honest LLM executors depart from their own plans. That rate sets a lower bound on how reliable selective translucency can be in practice.
The third piece is a leakage analysis. An attacker LLM sees an opponent's published policies and tries to infer that opponent's private budget. We compare the attacker's success when policies are published against its success when full traces are shared.
Baselines
The naive reference point is opaque decomposed agents. These use the same planner/executor agents with nothing published and no audit. This corresponds to the classical setting in which Parametric Open Source Games reports that closed-source agents "invariably converge to mutual defection" in the Prisoner's Dilemma. We expect the same outcome from LLM players, and confirming it calibrates the environment.
The second baseline is published plans with no audit ($p=0$). This is cheap talk, and it isolates whether the auditor is necessary at all.
The strongest existing competitor is full transparency. Each agent sees the opponent's complete planner and executor transcript, including private state. This is the LLM analogue of the open-source and simulation settings in Proving Your Way to Cooperation and Game Theory with Simulation of Other Players. It is the strongest competitor because, in principle, it gives the most information for conditional cooperation.
We run full transparency in two forms. The raw form passes the entire transcript. The filtered form passes only the most anomalous 20–25% of steps, selected with a reconstruction-based anomaly filter in the style of Detect Before You Attribute. This gives full transparency its best chance against long-context degradation.
The key ablation is a monolithic intention-stating agent. A single LLM sees both public and private state, publishes a stated intention, and then acts. It is audited at the same rate $p$. This isolates whether the decomposition matters. Our claim is that it does, for two reasons. The monolithic agent's stated intention is written with knowledge of its secrets, so it may leak them. It may also strategically shade the statement in a way a planner without private access cannot.
All conditions use the same base model, the same per-episode token budget, and the same number of prompt-tuning iterations, which is fixed in advance for every condition. All conditions also share the same seeds and opponent pools.
The literature in the file reports no comparable numbers for any of these settings. There is no reported result on Lacuna for cooperation rates of LLM agents under inspection or auditing. The only quantitative anchors are from different tasks. Detect Before You Attribute reports a 76% token reduction from filtering on the Who and When benchmark. ALYMPICS reports winner survival rates of 40% for round-one winners and 80% for round-two winners in the opaque water auction, and our opaque condition should reproduce that regime.
Beating full transparency means the following. Selective translucency must reach mutual cooperation and welfare that are statistically indistinguishable from, or higher than, the filtered full-transparency baseline. It must do so with a clearly smaller token cost and clearly lower budget-inference accuracy by the attacker. Beating the monolithic ablation means achieving equal cooperation with lower leakage.
Why not just…?
Why not just let agents announce their intentions (cheap talk)? Without an audit, a published plan is costless to violate. Under the payoff structure above, an adversarial executor should defect whenever the opponent trusts the plan. This is exactly the opacity problem that translucent-player theory addresses (Game theory with translucent players). The $p=0$ baseline tests this directly. If honest-prompted LLMs cooperate on cheap talk alone, that is informative. Against adversarial executors, however, we predict collapse, and the experiment would show it.
Why not just share full traces, perhaps with filtering? There are three concrete costs. First, raw traces degrade LLM judges and must be cut to about a fifth of their length (Detect Before You Attribute). Second, token exhaustion caused all 5 of the 45 proof failures in Proving Your Way to Cooperation. Third, exposing private state invites the privacy backfire proven in Game Theory with Simulation of Other Players.
Filtering is a real threat on cooperation alone, and it may match us there. The comparison is therefore settled on the joint result: cooperation, tokens, and the attacker's leakage accuracy together.
Why not just enforce the published program, so the environment executes it? Enforcement is the $p=1$ endpoint of our design, and where it is cheap it should be used. However, enforcement requires running the policy on every private input. For long, multi-step actions such as bids over 20 days or code edits, an executor is needed precisely because the policy does not fully determine the action.
The contribution here is the threshold law. It tells a designer that auditing a fraction $p^*$ of episodes suffices. If the measured threshold turns out to be near 1, this alternative wins, and we would report that.
Why not just use one agent that states its intention? This is our main ablation. If it matches the decomposed agent on both cooperation and leakage, then decomposition adds nothing, and the idea reduces to "audited intention statements." We would accept that result.
Proposed Experiments
Experiment 1: matrix-game threshold test. We use the canonical 2×2 suite from Parametric Open Source Games: Prisoner's Dilemma, Stag Hunt, Chicken, and Harsh PD. Players are LLM planner/executor pairs. Planners write policies in the bot language of the Lean OSGT library (Proving Your Way to Cooperation), and executors are adversarial. We sweep $p$ from 0 to 1. The decisive check is whether the empirical cooperation rate jumps near the payoff-derived threshold $p^*$, shifted by the measured honest-drift rate, in each game. If the jump occurs far from the predicted point, or there is no sharp transition, the core analogy fails.
Experiment 2: private-information auction. We extend the Water Allocation Challenge from ALYMPICS to 5 agents over 20 days. Each agent has a private budget and a private daily need. We add the option to publish planner policies under audit. The metrics are total survival (welfare), exploitation rate, tokens per episode, and the attacker's accuracy in inferring budgets from what is published. All baselines above run here.
The decisive check has two parts. Selective translucency must match filtered full transparency on welfare, while giving the attacker near-chance budget inference. The monolithic ablation must show higher leakage than selective translucency at equal welfare.
Experiment 3: drift and model scale. We repeat Experiment 2 with smaller open models. Self-collaboration Code Generation via ChatGPT reports that roles collapse below about 13B parameters. We therefore predict that honest drift rises at small scale and pushes the effective threshold toward 1. This would mark where the method stops being practical, and that is the result a practitioner most needs.
References
- Bayesian games with intentions
- Game Theory with Translucent Players
- Parametric Open Source Games
- Game Theory with Simulation of Other Players
- Proving Your Way to Cooperation: Formalizing Proof-Based Open Source Game Theory in Lean
- MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems
- Self-Collaboration Code Generation via ChatGPT
- Detect Before You Attribute: Cascade Failure Attribution for Multi-Agent Systems
- ALYMPICS: LLM Agents Meet Game Theory
- Game theory with translucent players