Time Spent Thinking in Online Chess Reflects the Value of Computation

Evan M. Russek, Daniel Acosta-kane, Bas van Opheusden, Marcelo G. Mattar, Thomas L. GriffithsCogn. Sci 2025 · DOI

TL;DR

This paper investigates how chess players allocate their thinking time, finding that players strategically invest more time when the potential benefit of deeper analysis is higher. This behavior is more pronounced in stronger players, suggesting that effective resource management is a key component of expertise.

What Problem Does This Paper Address?

Humans are remarkably good at making complex decisions despite limited cognitive resources. A key question is how we achieve this efficiency. Recent theories propose that we intelligently manage our "mental effort" by making meta-decisions about when and how much to compute. This means we should spend more time thinking when the expected payoff (better decision quality) outweighs the cost (time, energy). However, most research on this topic has used simplified lab tasks, making it hard to generalize to complex, real-world planning like chess. This paper aims to bridge that gap by analyzing a massive dataset of online chess games to see if human players exhibit this strategic resource allocation in a complex domain.

Methods

The researchers used a two-pronged approach:

  1. AI-powered "computation" measurement: They employed the Stockfish chess engine, a powerful AI, to quantify the "benefit of computation" for any given chess position. This was done by comparing the quality of a move identified by a shallow search (approximating a player's initial assessment) versus a deep search (approximating the outcome of extensive computation). The difference in board position advantage between these two outcomes served as the "benefit of computation."
  2. Large-scale behavioral data analysis: They analyzed over 12 million online chess games from Lichess.org. They measured players' "move times" (time spent thinking before making a move) and correlated this with the AI-calculated "benefit of computation" for each position. They also incorporated player Elo ratings to assess how expertise influences this relationship.

Datasets

  • Lichess.org dataset: A massive dataset of over 12 million online chess games played in 2019, containing move and timing information. This dataset was previously analyzed in other research projects.

Key Insights and Ideas

  • Thinking is proportional to potential gain: Players don't just think for a fixed amount of time; they adjust their thinking time based on how much better a move might become with more analysis. This suggests a sophisticated, albeit implicit, cost-benefit calculation.
  • Expertise sharpens strategic thinking: Stronger players show a more pronounced relationship between thinking time and the benefit of computation. This implies that developing expertise involves not just better move evaluation, but also better management of cognitive resources.
  • Beyond "oracle" knowledge: The study also explored "expected benefit of computation," which accounts for player uncertainty about the outcome of their analysis. This "expected benefit" was an even better predictor of thinking time than the "true benefit," suggesting players are sensitive to the potential value of computation, even when the exact outcome is unknown.
  • Cost of time matters: The research also investigated the cost of spending time in chess, finding that players' behavior aligns with a cost-structure that increases non-linearly with time spent and depends on the remaining time on the clock. This reinforces the idea that players are considering the trade-offs involved.

Key Results

  • Positive correlation: Move times were positively related to the benefit of computation (∆Uc) across all time-control settings. For example, in a 60-second game, a higher benefit of computation could lead to an increase of about 0.67 seconds in thinking time.
  • Concave relationship: The relationship between thinking time and the benefit of computation was concave, meaning the impact of additional thinking time diminished as the potential benefit increased.
  • Expertise effect: The influence of the benefit of computation on move times increased with player Elo rating. For instance, the interaction of Elo with the benefit of computation ranged from 0.02 to 3.07 seconds across different time controls.
  • Expected benefit is key: Move times were more strongly related to the expected benefit of computation (E[∆Uc]) than the true benefit (∆Uc), especially in situations where the true benefit was zero but the expected benefit was positive.

Limitations and Future Work

  • "All-or-none" computation model: The study models computation as an "all-or-none" discrete action. The authors acknowledge that human planning in chess is likely more incremental, involving partial computations. Future work could explore more fine-grained models of human planning.
  • Approximating human strategy: The study uses Stockfish, an AI, to approximate human decision-making. While robust, it's not a perfect model. Future research could explore how players learn to estimate the error of their pre-computation estimates.
  • Process-level hypotheses: While the study provides evidence for a computational-level hypothesis (players behave as if they are optimizing), it doesn't specify the exact algorithms or processes players use. Future work should investigate specific process-level hypotheses, such as feature-based estimation or heuristic strategies.
  • Discrepancies in time-left effects: The study noted a discrepancy between the model's prediction and human move times regarding the influence of time left on the clock, particularly at the start and end of games. This might be due to differences in how humans and AI handle openings.

Summary updated

BibTeX
@article{russek2025time,
  author = {Evan M. Russek and Daniel Acosta-kane and Bas van Opheusden and Marcelo G. Mattar and Thomas L. Griffiths},
  title = {{Time Spent Thinking in Online Chess Reflects the Value of Computation}},
  journal = {Cogn. Sci.},
  year = {2025},
  doi = {10.1111/cogs.70119},
}
Figure