Abstract
The chess domain is well-suited for creating an artificial intelligence (AI) system that mimics real-world challenges, including decision-making. Throughout the years, minimal attention has been paid to investigating insights derived from unstructured chess data sources. In this study, we examine the complicated relationships between multiple referenced moves in a chess-teaching textbook, and propose a novel method designed to encapsulate chess knowledge derived from move-action phrases. This study investigates the feasibility of using a modified sentiment analysis method as a means for evaluating chess moves based on text. Our proposed Aspect-Based Sentiment Analysis (ABSA) method represents an advancement in evaluating the sentiment associated with referenced chess moves. By extracting insights from move-action phrases, our approach aims to provide a more fine-grained and contextually aware 'chess move'-based sentiment classification. Through empirical experiments and analysis, we evaluate the performance of our fine-tuned ABSA model, presenting results that confirm the efficiency of our approach in advancing aspect-based sentiment classification within the chess domain. This research contributes to the area of game-playing by machines and shows the practical applicability of leveraging NLP techniques to understand the context of strategic games.
Introduction
Modern chess engines like Stockfish and AlphaZero have achieved superhuman performance through brute-force calculation and deep reinforcement learning. However, these systems often function as "black boxes," providing numerical evaluations without the strategic intuition or linguistic explanations that human players use to learn. Humans typically acquire chess knowledge through textbooks, where moves are described using qualitative predicates (e.g., "White should attack the center").
This paper introduces ASSESS (Aspect-based Sentiment Evaluation of Chess Moves), a novel method that leverages Natural Language Processing (NLP) to bridge the gap between human strategic commentary and machine-readable move evaluations. By redefining a chess "aspect" as a triple of Player-Predicate-Move, the system can translate human strategic commentary into quantitative move evaluations that correlate with top-tier chess engines.
The ASSESS Methodology
The core innovation of ASSESS lies in its definition of a chess "aspect." Traditional sentiment analysis often fails in chess contexts because a single sentence may discuss multiple moves with opposing evaluations (e.g., "White should play e4 instead of d4"). To address this, the researchers apply Aspect-Based Sentiment Analysis (ABSA), allowing the model to assign different sentiments to different moves within the same sentence.
The methodology incorporates three distinct architectural approaches to aspect representation:
- Move Only: Treating the move notation (e.g., "e5") as the sole aspect.
- Move-Action: Including the player and the specific verb (predicate) associated with the move.
- Move-Action with Clustering: Enhancing the predicate with semantic categories (e.g., Attack, Capture, Defend) derived from WordNet and spaCy.
By fine-tuning a RoBERTa-base model (FT-RB) on domain-specific data, the system learns to assign sentiments (Positive, Negative, or Neutral) to these move-action phrases based on the surrounding context.
Dataset and Annotation
To train and validate the model, the authors utilized the LEAP corpus and created a new gold-standard dataset consisting of 726 sentences manually annotated from a chess textbook. Each entry identifies the move, the player, the predicate, and the intended sentiment.
To handle class imbalance—as textbooks naturally focus on "good" moves—the researchers used back-translation (English → German → English) to synthetically double the number of negative and neutral examples.
Experimental Results
The empirical results demonstrate that the inclusion of the predicate is vital for accuracy. The fine-tuned RoBERTa model using the full move-action phrase reached an F1-score of 62%, a significant jump over the 55% achieved by looking at the "move only" aspect.
Key insights from the results include:
- Sentiment as Game State: There is a measurable correlation between human "sentiment" in text and objective "win probability" from engines. Positive sentiment aligns with high win probability, while Neutral sentiment often aligns with a "Draw" evaluation from Stockfish.
- Minority Class Improvement: By using oversampled, augmented data, the model's ability to correctly identify "Negative" and "Neutral" moves improved by 20% to 30%.
Conclusion and Future Work
ASSESS demonstrates the feasibility of using NLP to extract strategic insights from unstructured text. While the current model excels at explicit move-action phrases, future hurdles include handling implicit moves and integrating multimodal grounding—training the model to look at the text and the actual board state (FEN notation) simultaneously to ensure the sentiment matches the physical reality of the pieces.
Summary updated