Real-Time Parallel Counterfactual Regret Minimization

Boning Li, Longbo HuangarXiv 2026 · arXiv

TL;DR

Parallel CFR is a novel parallelization framework for real-time, depth-limited Counterfactual Regret Minimization (CFR) that splits each iteration into a seven-stage heterogeneous CPU-GPU pipeline. By parallelizing across both information sets and tree nodes, it achieves a 3.3–3.4× speedup on postflop streets, enabling consumer-grade desktop hardware to compute strong approximate equilibrium strategies within tight real-time decision windows.

What Problem Does This Paper Address?

In imperfect-information games like Texas Hold'em poker, AI systems must make strategic decisions without knowing the opponent's exact state. The dominant algorithm for solving these games is Counterfactual Regret Minimization (CFR). In real-time settings, a game-playing bot has only a few seconds per turn to compute its strategy, and its playing strength is directly tied to how many CFR iterations it can complete within that brief window.

While modern game-solving stacks achieve efficiency by combining depth-limited solving, abstraction, and online pruning, this heterogeneous mix resists simple parallelization. Prior parallel CFR techniques were designed for simpler tabular variants and cannot handle these modern components. As a result, previous superhuman poker AIs (like Libratus) had to rely on massive supercomputers during live play. This paper addresses the challenge of making real-time, state-of-the-art CFR solving highly parallel and executable on a single desktop-class machine.

Methods

The authors introduce Parallel CFR, which systematically exploits the internal parallelism of modern game-solving components without introducing any numerical approximation errors.

The core architecture relies on a few key innovations:

  • Seven-Stage Pipeline: Each CFR iteration is decomposed into seven distinct stages, allowing each phase to use the parallelization strategy best suited to its data dependencies.
  • Two Dimensions of Parallelism: The framework parallelizes computation by information set (vectorizing over hands within a set) and by tree node (processing disjoint subtrees concurrently).
  • Heterogeneous CPU-GPU Execution: Tree operations, reach probability aggregations, and showdown equity calculations are handled by the CPU via multi-threaded OpenMP execution. Meanwhile, non-terminal leaf node evaluations are batched into a single forward pass and offloaded to the GPU via neural network inference.
  • Parallel Middle Pass Forking: As shown in the pipeline diagram below, after Stage 2, the pipeline forks. The CPU handles opponent reach and showdown equity (Stages 3 and 4) while the GPU simultaneously executes leaf evaluations (Stage 5), effectively masking GPU overhead on postflop streets.
Figure 1: Pipeline of a single CFR iteration in Parallel CFR. Blue stages execute on CPU with OpenMP parallelism; the orange stage runs on GPU. Stages 3/4 and Stage 5 execute in parallel (dashed box). Annotations indicate the parallelization dimension.
  • Multiplicative Pruning: The framework natively integrates online pruning (specifically the EVPA framework). Because pruning eliminates dominated actions before CFR iterations begin, it reduces the effective tree size and acts as a multiplicative speedup across all subsequent pipeline stages.

Datasets

The framework was evaluated on Heads-Up No-Limit Texas Hold'em (HUNL) with depth-limited solving applied to the end of each betting round. To test scalability, the authors varied the tree sizes by adjusting the stack-to-pot ratio (SPR $\in \{4, 16, 64\}$) and the number of raise actions ($|A_{raise}| \in \{2, 3\}$).

The primary large-scale benchmark configuration consisted of a massive depth-limited game tree with an SPR of 64 and 3 raise actions, encompassing 2,006,256 information sets and over 1.08 billion histories.

Key Insights and Ideas

  • Information-Set Granularity: Unlike prior parallelization attempts that tied granularity to tree depth or public nodes, Parallel CFR parallelizes across independent information sets. This aligns perfectly with the natural unit of computation in CFR.
  • GPU Masking: On postflop streets (Flop and Turn), the GPU leaf evaluation time is entirely masked by the longer CPU branch running in parallel. This creates a highly efficient balance between the two hardware types.
  • Linear Scaling with Tree Size: The pipeline's execution time scales approximately linearly with the number of tree nodes. This indicates that the parallelization overhead is minimal and the system maintains high parallel efficiency even as the game tree grows 15-fold.

Key Results

The authors ran their experiments on a single desktop-class device (an NVIDIA DGX Spark equipped with a GB10 Grace Blackwell superchip).

Key performance metrics include:

  • Postflop Speedups: Parallel CFR achieved a 3.3× speedup on the flop and turn, and a 3.4× speedup on the river compared to a single-threaded baseline.
  • Absolute Latency: Per-iteration pipeline time was brought down to just 47–54 ms on postflop streets for a tree with over 1 billion histories, allowing the solver to easily complete ~100 iterations within a standard 5-second real-time decision budget.
Table 2: Per-iteration CFR pipeline time on HUNL.
  • Lower Exploitability: In a direct convergence comparison against PokerRL (a highly efficient open-source vectorized solver), Parallel CFR achieved ~7× lower exploitability under the exact same wall-clock time budget across multiple CFR variants (DCFR and Predictive CFR+).

Limitations and Future Work

While Parallel CFR significantly advances single-device performance, the authors note a few limitations and areas for future exploration:

  • Single-Instance Focus: The current framework optimizes a single CFR solve instance. Combining this method with distributed computing across multiple machines could yield even greater scaling.
  • CPU Bottlenecks: Memory bandwidth becomes a bottleneck when multiple CPU threads compete for the shared L3 cache.
  • Future Directions: The authors suggest that developing a fully GPU-native implementation of the entire pipeline and introducing asynchronous CPU-GPU overlap could further minimize per-iteration latency.

Summary updated

Abstract

Counterfactual Regret Minimization (CFR) is the dominant algorithmic family for solving large imperfect-information games, underpinning breakthroughs such as Libratus and Pluribus in No-Limit Texas Hold'em poker. In real-time game-playing systems, the solver must compute a near-equilibrium strategy within a strict time budget of only a few seconds per decision, and the number of CFR iterations completed in this window directly determines play strength. We present \textbf{Parallel CFR}, the first parallelization framework for real-time depth-limited CFR solving that seamlessly integrates pruning, abstraction, and advanced CFR variants. We decompose each CFR iteration into a pipeline of seven stages and identify two orthogonal dimensions of parallelism: \emph{by information set} and \emph{by tree node}. Leaf node evaluation is offloaded to GPUs via batched neural network inference, creating a heterogeneous CPU--GPU pipeline. Experiments on Heads-Up No-Limit Texas Hold'em demonstrate that Parallel CFR achieves $3.3$--$3.4\times$ speedup over the single-threaded baseline on postflop streets, with per-iteration time of ${\sim}47$--$54$~ms on a depth-limited game tree with over $1$ billion histories. All experiments run on a single desktop-class device (NVIDIA DGX Spark), enabling hundreds of CFR iterations within a typical real-time decision budget without requiring datacenter-scale infrastructure.
BibTeX
@misc{li2026real,
  author = {Boning Li and Longbo Huang},
  title = {{Real-Time Parallel Counterfactual Regret Minimization}},
  year = {2026},
  eprint = {2605.19928},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.19928},
}
Figure