Improving Search with Supervised Learning in Trick-Based Card Games

Christopher Solinas, Douglas Rebstock, Michael Buro

Abstract

In trick-taking card games, a two-step process of state sampling and evaluation is widely used to approximate move values. While the evaluation component is vital, the accuracy of move value estimates is also fundamentally linked to how well the sampling distribution corresponds the true distribution. Despite this, recent work in trick-taking card game AI has mainly focused on improving evaluation algorithms with limited work on improving sampling. In this paper, we focus on the effect of sampling on the strength of a player and propose a novel method of sampling more realistic states given move history. In particular, we use predictions about locations of individual cards made by a deep neural network — trained on data from human gameplay — in order to sample likely worlds for evaluation. This technique, used in conjunction with Perfect Information Monte Carlo (PIMC) search, provides a substantial increase in cardplay strength in the popular trick-taking card game of Skat.

Introduction

In imperfect information games like Bridge or Skat, players must make decisions without knowing the exact cards held by opponents. A common approach is Perfect Information Monte Carlo (PIMC) search, which involves sampling several possible "worlds" (hidden card distributions), solving each as if all cards were visible, and choosing the move that performs best on average.

However, traditional PIMC often relies on uniform random sampling, which ignores the strategic implications of previous moves. Human play is non-random; every card played reveals information about a player's hand and strategy. If an agent samples unrealistic or "impossible" worlds, its search results become inaccurate. This research addresses this gap by using supervised learning to infer more realistic state distributions based on the entire history of moves.

Methodology: PIMC with State Inference

The authors propose a technique called PIMC with state inference (PIMC+). The core innovation is using a Deep Neural Network (DNN) to perform "Individual Card Inference."

The deep neural network architecture used for individual card inference, incorporating move history and state features.

Instead of predicting the entire game state, the network predicts the probability of each of the 32 cards being in one of four locations (Player 1, Player 2, Player 3, or the "skat" discard pile). The model processes bidding history, the cards currently on the table, "void" suits, and—crucially—the entire cardplay history. By assuming card locations are independent for the sake of probability calculation, the agent can assign a "likelihood score" to any sampled state, guiding the PIMC search toward worlds that are consistent with actual human play patterns.

Results and Insights

The model was trained on a massive dataset of 20 million human games from the DOSKV server. Evaluation on a holdout set and simulated tournaments against state-of-the-art bots like "Kermit" revealed several key findings:

  • The Mid-Game Sweet Spot: Inference is most effective during tricks 3 through 5. Early in the game, information is too sparse to narrow down hands; late in the game, the number of possible states naturally collapses, making inference less critical.
  • Asymmetric Information: It is significantly easier for "defenders" to infer the "soloist's" hand than vice versa. This occurs because the soloist chooses the game type based on their cards, effectively "telegraphing" their strength from the start.
Information gain across different game types (Null, Grand, Suit) for Defenders and Soloists, highlighting the mid-game peak in inference utility.
  • Performance Gains: The proposed agent, BDCI, demonstrated a statistically significant increase in points over previous methods. In "suit" and "null" games, the point spread compared to no-inference baselines was substantial, with deltas reaching over +10 points in some configurations.
  • Efficiency: Despite the DNN overhead, the agent maintains a decision time of approximately 0.3 seconds per move, making it fast enough for real-time play against humans.

Conclusion

By leveraging deep learning to "read" opponents based on move history, this work significantly improves the accuracy of state sampling in PIMC search. While the current model uses an independence assumption for card locations, future work could involve enforcing stricter hand-size constraints within the network architecture and accounting for deceptive or "psychological" play styles. Integrating this inference technique with advanced search algorithms like Information Set Monte Carlo Tree Search (ISMCTS) remains a promising avenue for further refining trick-taking AI.

Summary updated

Abstract

In trick-taking card games, a two-step process of state sampling and evaluation is widely used to approximate move values. While the evaluation component is vital, the accuracy of move value estimates is also fundamentally linked to how well the sampling distribution corresponds the true distribution. Despite this, recent work in trick-taking card game AI has mainly focused on improving evaluation algorithms with limited work on improving sampling. In this paper, we focus on the effect of sampling on the strength of a player and propose a novel method of sampling more realistic states given move history. In particular, we use predictions about locations of individual cards made by a deep neural network — trained on data from human gameplay — in order to sample likely worlds for evaluation. This technique, used in conjunction with Perfect Information Monte Carlo (PIMC) search, provides a substantial increase in cardplay strength in the popular trick-taking card game of Skat.
BibTeX
@inproceedings{solinas2019improving,
  author = {Christopher Solinas and Douglas Rebstock and Michael Buro},
  title = {{Improving Search with Supervised Learning in Trick-Based Card Games}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  url = {https://openreview.net/forum?id=vesAYILOeH},
}

Loading related content...

Figure