A poker agent that plays a Nash equilibrium, a strategy no opponent can beat in expectation, is safe but leaves money on the table against weak players. An agent that best-responds to a learned model of its opponent wins more when the model is right, and can be badly beaten when it is wrong. This is the tension that algorithms-with-predictions research calls the consistency–robustness trade-off. Consistency is how well you do when the prediction is correct. Robustness is the worst case when it is not. Recent game-playing methods let an agent partly trust an opponent model while bounding how exploitable it becomes. All of them treat the prediction as a single object with one trust knob. None asks a question that online-algorithms research treats as basic: how much advice is needed, and does more advice always help?
We propose to study the depth of opponent advice as the central variable. The advice might be only which hands the opponent is likely to hold when a sub-game begins. It might add a forecast of the opponent's next move, of its next several moves, or its entire policy. We will compute exact safety–exploitation frontiers for each depth in small poker and bluffing games, and test whether deeper advice stops paying off once the predictor is noisy. Game search already has a known version of this effect: looking deeper can make an agent play worse. We then build a sub-game solver that assigns a separate trust level to each depth, set online from how accurate the predictor has been at that depth, while never becoming more exploitable than the precomputed strategy it starts from.
Core Claim
In an extensive-form game (a game tree where players act in turn and some information is hidden), a prediction about the opponent is not one number. It is a collection of forecasts at different depths of the tree, and their errors differ systematically. Deep forecasts are fitted on rarer situations and are noisier. Deep deviations are also where a deceptive opponent can set traps. Our claim has two parts. First, the achievable trade-off between consistency and robustness depends strongly on which depths of advice the agent is allowed to use. Second, once exploitability is held fixed, value against the real opponent need not increase with advice depth under realistic predictor error. We call this an "advice-depth pathology." The practical consequence is that trust should be scheduled by depth, from measured per-depth error, rather than set by one global weight.
The closest existing work motivates this directly. Safe and Robust Subgame Exploitation in Imperfect Information Games (OX-Search) enforces "adaptation safety," meaning the adapted strategy may be no more exploitable than the blueprint (the precomputed approximate equilibrium). It uses only the opponent's reach probabilities at the sub-game root, and it lists the use of predictions about future actions inside the sub-game as open. That is precisely the depth-0 end of our axis. Safe Opponent-Exploitation Subgame Refinement (SES) mixes a safety branch and an exploitation branch with one fixed weight $\alpha$. Agents That Certify Their Own Exploits (CS-RNR) uses a single "pin level" $p$: the probability that the opponent is forced to follow the model. It chooses $p$ by exactly auditing exploitability, and it reports that sparse, deep situations starve its statistical tests of data. We keep the restricted-response mechanism of CS-RNR and the safety definition of OX-Search. What we add is structure over depth, together with exact frontier measurements that show whether that structure matters.
Research Program
The object we study is a depth-scheduled restricted response. Let $\hat\sigma_2$ be the predicted opponent policy and $d(I)$ the number of opponent decisions below the sub-game root at which information set $I$ occurs. An information set is a set of game states the acting player cannot tell apart. In the restricted game, the opponent at $I$ is forced to play $\hat\sigma_2(I)$ with probability $p_{d(I)}$ and is otherwise free to play adversarially. Our strategy $x_{\mathbf p}$ is the equilibrium response in this restricted game, where $\mathbf p=(p_0,p_1,\dots)$ is the vector of per-depth pins. Its consistency is $u(x_{\mathbf p},\hat\sigma_2)$, the expected payoff if the model is exactly right. Its robustness is its exploitability $e(x_{\mathbf p})$, the most a best-responding opponent can win against it. The frontier for an advice class $\mathcal A$ is
$$F_{\mathcal A}(\epsilon)=\max_{\mathbf p\in\mathcal A}\; u(x_{\mathbf p},\sigma_2^{*})\quad\text{s.t.}\quad e(x_{\mathbf p})\le e(\sigma^{bp})+\epsilon,$$
where $\sigma_2^{*}$ is the true opponent, $\sigma^{bp}$ the blueprint, and $\epsilon\ge 0$ an extra exploitability budget. The advice class $\mathcal A_D$ allows nonzero pins only at depths below $D$. Depth $D=0$ roughly corresponds to reach-only advice, and $D=\infty$ is a full-policy restricted response. Measuring value against the true opponent, rather than against the model, is what lets depth show a cost.
The program has three parts.
The first part computes exact frontiers in small games. Kuhn poker (12 information sets), Leduc Hold'em (288), and 5-rank Leduc (780) are small enough to solve restricted games by sequence-form linear programming and to compute exploitability exactly. We will trace $F_{\mathcal A_D}$ for each $D$ across controlled predictor error. For Kuhn poker we will attempt closed-form piecewise-linear curves by parametric linear programming. These would be the first exact consistency–robustness curves in an extensive-form game, analogous to the ski-rental curves of Optimal Robustness-Consistency Trade-offs for Learning-Augmented Online Algorithms.
The second part is theory. We aim for two results. One is a separation construction: a family of small games in which deeper advice strictly enlarges the frontier when advice is exact, but strictly shrinks value at fixed $\epsilon$ once per-depth error passes a threshold. This is the prediction-side counterpart of the lookahead pathology of Limited Lookahead in Imperfect-Information Games, which assumes the opponent's heuristic is known exactly. The other is an advice-compression result: conditions under which reach-only advice already achieves consistency within a stated gap of full-policy advice. This echoes the open question in Online Conversion with Switching Costs, where exact consistency needs advice that grows with the time horizon.
The third part is an online algorithm. A predictor is trained on observed hands. We estimate per-depth error with held-out likelihood and anytime-valid confidence sequences, as in CS-RNR, but pooled by depth rather than tested separately at every information set. Pooling is our answer to CS-RNR's confirmation starvation. We then choose $\mathbf p$ so that pins shrink where error is high. Safety is enforced by certify-and-back-off in the small games, and by OX-Search-style margin constraints inside real-time sub-game solving in Flop Hold'em, where exact full-tree audits are infeasible. Because Safe Test-Time Reinforcement learning for Imperfect Information Games shows that equal-value sub-game solutions can differ widely in full-game exploitability, we will adopt its blueprint-tremble refinement when resolving.
Baselines
The naive reference point is the blueprint equilibrium, which ignores predictions. It sets the exploitability floor and the zero-gain line. Its opposite is an unconstrained best response to the same predictor. That shows the maximal consistency and the exploitability cost of trusting blindly.
Among existing methods, OX-Search is the strongest competitor on this question. It is the method that already guarantees adaptation safety relative to the blueprint, it reports beating SES and restricted Nash response on Leduc, Flop Hold'em, Liar's Dice and Goofspiel, and it is exactly our depth-0 point. We find no reported number on Lacuna for OX-Search's return or exploitability (the file gives qualitative and figure-only results). We will therefore rerun it ourselves with the same blueprint, predictor, sub-game boundaries and equal CPU time per decision, and tune its multiplier $\beta$ over the same grid size we allow for our pins.
CS-RNR is the strongest global-pin restricted response. The reported figures we can anchor to are its gain of 6.2 times the steady-state gain of a money-verified binary gate on Leduc and 5-rank Leduc, zero certificate violations across 36,000 audited hands, and an exploitability of 0.15 for its restricted solve against 2.04 for a trajectory mixture at the same pin level CS-RNR. We will reproduce it on Leduc with its own confidence-sequence settings and the same risk budget, so its 6.2× figure can be checked in our pipeline. SES is run with $\alpha$ tuned on the same grid; we found no reported numeric result on Lacuna for SES.
The key ablation isolates depth structure. It compares depth-scheduled pins against the best single global pin, and against per-information-set pins set from observation counts, each with the same tuning budget and the same certification.
To claim a win, three things must hold together. At matched exploitability (at or below the blueprint's), the depth-scheduled method must earn higher expected value per hand against held-out opponents than tuned OX-Search and tuned CS-RNR. It must beat the best global pin by more than the seed-to-seed confidence interval. And it must show no certificate violations.
Why not just…?
Why not just tune one global trust weight well? The global pin is a special case of per-depth pins, so our frontier weakly dominates it by construction. The real question is whether the gap is large enough to matter and whether it can be exploited online. This is a genuine threat. If the exact frontiers in the small games show the best global pin within noise of the best depth schedule, the depth-scheduled algorithm is not worth building. We would then report the negative result and keep the frontier measurements as the contribution. The ablation described above settles this before any large-game work.
Why not just use reach-only advice, as OX-Search does? It is possible that most exploitable value sits in the opponent's hand distribution at the sub-game root. If so, deeper advice is unnecessary, and that would be the advice-compression result, not a failure. The frontier comparison between $\mathcal A_0$ and $\mathcal A_\infty$ under realistic learned-predictor error measures this directly, per game. We expect betting-heavy Leduc and Liar's Dice, where opponents misplay later decisions, to be the likeliest places for deeper advice to pay off.
Why not just set per-information-set trust from visit counts, or test each set separately as CS-RNR does? Visit counts measure how much data exists, not how wrong a learned predictor is. Per-set testing is what produces CS-RNR's confirmation starvation at rare, deep information sets. Pooling error estimates by depth trades resolution for statistical power. The count-based ablation shows which side of that trade wins. If count-based trust matches ours, depth is not the right grouping, and we will say so.
Why not just mix the equilibrium and the best response hand by hand? CS-RNR measured this in Leduc: a trajectory mixture reached exploitability 2.04, 13.6 times its budget, while a restricted solve stayed at 0.15. The same point appears in online matching, where Learning-Augmented Online Bipartite Fractional Matching shows that integrating advice beats coin-flip mixtures. We include mixtures only as a sanity baseline.
Proposed Experiments and Decisive Checks
All games come from OpenSpiel: Kuhn, Leduc, 5-rank Leduc, Liar's Dice (1,4) and (2,4), Goofspiel 5, and Flop Hold'em for scale. Robustness is measured by exact exploitability (OpenSpiel's NashConv, a standard exploitability-style measure) in the small games. In Flop Hold'em, where exact computation is infeasible, we use a local best-response approximation.
Opponents come in three types. The first is shuffled equilibria at the $Pr_{shuffle}=0.3$ and $0.7$ levels used by SES. The second is scripted biased opponents whose mistakes sit at controlled depths. The third is a deceptive "teaching" opponent that matches the model early and then switches to a counter-exploit, which directly probes the failure mode OX-Search describes. Predictors range from exact, through noise injected per depth, to a small network trained on 100 to 10,000 observed hands, so that error grows with depth naturally.
The first decisive check runs on exact small-game frontiers. It asks whether the value-at-fixed-exploitability curve for $\mathcal A_D$ stops increasing in $D$ for learned predictors while remaining monotone for exact predictors. If it is monotone in every game and every opponent, the pathology hypothesis is false.
The second check asks whether depth-scheduled pins, chosen online, sit measurably above the best global pin and above tuned OX-Search at matched exploitability, across at least three games and ten seeds.
The third check is safety. There must be zero certificate violations in the small games. In Flop Hold'em, measured exploitability must not exceed the blueprint's, including against the teaching opponent. Together these three checks decide whether advice depth is a real axis of the consistency–robustness trade-off in hidden-information games or merely a finer knob on an existing one.
References
- Safe and Robust Subgame Exploitation in Imperfect Information Games
- Safe Opponent-Exploitation Subgame Refinement
- Agents That Certify Their Own Exploits: Confidence-Scheduled Restricted Responses for Safe Opponent Exploitation
- Optimal Robustness-Consistency Trade-offs for Learning-Augmented Online Algorithms
- Limited Lookahead in Imperfect-Information Games
- Online Conversion with Switching Costs: Robust and Learning-Augmented Algorithms
- Safe Test-Time Reinforcement learning for Imperfect Information Games
- Learning-Augmented Online Bipartite Fractional Matching
- OpenSpiel: A Framework for Reinforcement Learning in Games