Learning to play the Chess Variant Crazyhouse above World Champion Level with Deep Neural Networks and Human Data

Johannes Czech, Moritz Willig, Alena Beyer, Kristian Kersting, Johannes Fürnkranz

TL;DR

This paper introduces CrazyAra, a deep neural network-based engine for the complex chess variant Crazyhouse. By leveraging supervised learning on human game data and optimizing Monte-Carlo Tree Search (MCTS), CrazyAra achieves world-champion level play, outperforming even the reigning human champion. This work demonstrates the effectiveness of efficient deep learning approaches for challenging games with limited data.

What Problem Does This Paper Address?

The authors tackle the challenge of building a strong AI for Crazyhouse, a chess variant with a significantly higher branching factor than standard chess due to the "drop" mechanic (reintroducing captured pieces). While deep learning approaches like AlphaZero have achieved superhuman performance in games like Go and chess using reinforcement learning from scratch, this method is computationally expensive and requires vast amounts of data. For Crazyhouse, the available human data is of lower quality and quantity compared to Go or chess, making a direct application of AlphaZero's zero-knowledge approach difficult. The paper aims to develop a highly competitive Crazyhouse engine using supervised learning on existing human data, focusing on efficiency and overcoming the unique challenges of the game.

Methods

CrazyAra employs a dual architecture: a deep neural network for evaluating board states and predicting moves, integrated with a Monte-Carlo Tree Search (MCTS) algorithm for decision-making. Key methodological contributions include:

  • Compact Input Representation: A fully Markovian board representation is used, removing historical game states to simplify the input and improve efficiency.
  • Data Normalization: A crucial preprocessing step involving rescaling and normalizing input features is introduced, significantly improving training performance.
  • Efficient Neural Network Architecture: A novel, more compact neural network design is proposed, incorporating advancements like grouped depthwise convolutions, pre-activation ResNets, and Squeeze-Excitation layers.
  • Sample-Efficient MCTS: Modifications to the MCTS algorithm are implemented to enhance its efficiency and reduce blunders. These include using Q-values for move selection, a transposition table for sharing evaluations, and techniques to avoid missing critical moves.
  • Supervised Learning: The neural network is primarily trained on a large dataset of human Crazyhouse games, a departure from the zero-knowledge reinforcement learning of AlphaZero.

Datasets

  • Lichess.org Human Games: The primary training dataset consists of 569,537 human games played on lichess.org between January 2016 and June 2018, with players having an Elo rating of at least 2,000.
  • Stockfish Self-Play Games: An additional dataset of 121,571 games generated by Stockfish 10 was used to train a variant called CrazyAraFish.

Key Insights and Ideas

  • Supervised Learning is Viable for Complex Games: The success of CrazyAra demonstrates that even for highly tactical games like Crazyhouse with limited data, supervised learning on human games can yield superhuman performance, offering a more computationally efficient alternative to pure reinforcement learning.
  • Data Normalization is Critical: The paper highlights that simple input normalization can lead to significant improvements in training speed and final accuracy, a crucial insight for efficient deep learning in games.
  • MCTS Can Be Made More Robust: By integrating Q-values and other enhancements, MCTS can be made more sample-efficient and less prone to blunders, making it a powerful tool even in games with high branching factors.
  • Crazyhouse as a Testing Ground: The game's tactical nature and reduced draw rate make it an excellent environment for testing and reinforcing the effects of search algorithm modifications.

Key Results

  • Superhuman Performance: CrazyAra achieved a 4-to-1 win rate against Justin Tan, the 2017 Crazyhouse World Champion, who is rated over 400 Elo higher than the average player in the training set.
  • Tournament Success: CrazyAra won against twelve out of thirteen participants in the second Crazyhouse Computer Championships 2017.
  • Move Prediction Accuracy: The engine achieved a move prediction accuracy of 60.4% after training on human games for 1.5 days.
  • CrazyAraFish Performance: In matches against Stockfish 10, CrazyAraFish won three games and drew one out of ten long-time control matches.
  • Efficiency Gains: The proposed neural network architecture and MCTS modifications led to improved efficiency and performance without requiring extreme computational resources.

Limitations and Future Work

  • Data Quality: While successful, the authors acknowledge that the human data used for training is of lower quality compared to what might be available for Go or chess, suggesting that further improvements could be made with higher quality data.
  • Reinforcement Learning Potential: The paper suggests that applying reinforcement learning (self-play) could further enhance CrazyAra's playing strength.
  • Network Design and MCTS Optimizations: Future work could explore faster rollout generation (e.g., using lower precision inference) and further improvements in network design and MCTS to boost performance.
  • Generalization: While CrazyAra performs exceptionally well, the authors note that the scheme could be further accelerated and improved in different areas for Crazyhouse.

Summary updated

Abstract

Deep neural networks have been successfully applied in learning the board games Go, chess and shogi without prior knowledge by making use of reinforcement learning. Although starting from zero knowledge has been shown to yield impressive results, it is associated with high computationally costs especially for complex games. With this paper, we present CrazyAra which is a neural network based engine solely trained in supervised manner for the chess variant crazyhouse. Crazyhouse is a game with a higher branching factor than chess and there is only limited data of lower quality available compared to AlphaGo. Therefore, we focus on improving efficiency in multiple aspects while relying on low computational resources. These improvements include modifications in the neural network design and training configuration, the introduction of a data normalization step and a more sample efficient Monte-Carlo tree search which has a lower chance to blunder. After training on 569,537 human games for 1.5 days we achieve a move prediction accuracy of 60.4%. During development, versions of CrazyAra played professional human players. Most notably, CrazyAra achieved a four to one win over 2017 crazyhouse world champion Justin Tan (aka LM Jann Lee) who is more than 400 Elo higher rated compared to the average player in our training set. Furthermore, we test the playing strength of CrazyAra on CPU against all participants of the second Crazyhouse Computer Championships 2017, winning against twelve of the thirteen participants. Finally, for CrazyAraFish we continue training our model on generated engine games. In ten long-time control matches playing Stockfish 10, CrazyAraFish wins three games and draws one out of ten matches.
BibTeX
@article{czech2020learning,
  author = {Johannes Czech and Moritz Willig and Alena Beyer and Kristian Kersting and Johannes Fürnkranz},
  title = {{Learning to play the Chess Variant Crazyhouse above World Champion Level with Deep Neural Networks and Human Data}},
  journal = {Frontiers Artif. Intell.},
  year = {2020},
  doi = {10.3389/frai.2020.00024},
}
Figure