Monkey Plays Pac-Man with Compositional Strategies and Hierarchical Decision-making

Qianli Yang, Zhongqiao Lin, Wenyi Zhang, Jianshu Li, Xiyuan Chen, Jiaqi Zhang, Tianming YangbioRxiv 2021 · DOI

TL;DR

Macaque monkeys can learn to play Pac-Man, exhibiting complex decision-making strategies that can be modeled as a hierarchical system. This research demonstrates that monkeys employ a "take-the-best" heuristic, dynamically switching between simple strategies and combining them to tackle the game's challenges, offering a new window into animal cognition.

What Problem Does This Paper Address?

Humans often simplify complex tasks by developing and applying strategies. While animals exhibit complex behaviors, demonstrating sophisticated, flexible strategy use in a computationally challenging task has been difficult to quantify. This paper aims to bridge this gap by investigating whether non-human primates can employ hierarchical and compositional strategies to solve a complex, dynamic problem, and to develop a computational model that accurately describes this behavior.

Methods

The researchers trained two macaque monkeys to play a modified version of the classic video game Pac-Man. They developed a "dynamic compositional strategy model" to analyze the monkeys' decision-making. This model breaks down the game into several "basis strategies" (e.g., moving towards the nearest pellet, avoiding ghosts). The model then estimates the "weights" of these strategies over time, reflecting which strategy is dominant at any given moment. This approach allows for the analysis of how monkeys combine these basic strategies into more complex, "compound strategies" to handle specific game situations. They also used eye-tracking and pupil dilation measurements to correlate physiological responses with strategy shifts.

Datasets

The study used data collected from two macaque monkeys (Monkey O and Monkey P) playing the adapted Pac-Man game over multiple sessions. The paper does not introduce a new benchmark dataset but rather uses the collected gameplay data for model fitting and validation.

Key Insights and Ideas

  • Monkeys as Pac-Man Masters: Macaque monkeys can learn to play Pac-Man, demonstrating a level of strategic complexity previously unquantified in such tasks.
  • "Take-the-Best" Heuristic in Primates: The monkeys appear to adopt a "take-the-best" heuristic, meaning they often rely on a single, dominant strategy at a time, simplifying decision-making.
  • Hierarchical and Compositional Strategies: Monkeys don't just use simple rules; they combine basic strategies into more complex ones to deal with different game scenarios, suggesting a hierarchical decision-making process.
  • Dynamic Strategy Switching: The monkeys dynamically switch between strategies based on the game state (e.g., number of pellets, ghost proximity), indicating adaptive behavior.
  • Physiological Correlates of Strategy Shifts: Pupil dilation significantly increases around strategy transitions, suggesting a cognitive cost associated with switching strategies.

Key Results

  • The dynamic compositional strategy model achieved over 90% accuracy in predicting the monkeys' decisions.
  • The model revealed that monkeys predominantly used one strategy at a time, with the "local" strategy being most common in the early game and "local" and "global" strategies being important in the late game.
  • Specific compound strategies were identified, such as "planned attack" (strategically hunting ghosts after eating an energizer) and "suicide" (intentionally getting caught by a ghost to reset the game when pellets are scarce).
  • Eye-tracking data showed that monkeys focused on different game elements depending on the strategy being employed.

Limitations and Future Work

  • Hand-crafted Strategies: The basis strategies in the model were hand-crafted, meaning they are based on human intuition about the game. Future work could explore more data-driven approaches to strategy discovery.
  • Generalizability: While the model fits the monkeys' behavior well, further research is needed to understand the neural mechanisms underlying these strategies and to see if similar strategies are employed in other complex tasks.
  • Neural Basis: The paper lays the groundwork for future experiments to investigate the neural circuits involved in strategy selection and switching.
  • Broader Cognitive Abilities: The Pac-Man paradigm offers a rich platform for studying other advanced cognitive abilities in animals, such as planning, learning, and social cognition.

Summary updated

Abstract

Abstract Humans can often handle daunting tasks with ease by developing a set of strategies to reduce decision making into simpler problems. The ability to use heuristic strategies demands an advanced level of intelligence and has not been demonstrated in animals. Here, we trained macaque monkeys to play the classic video game Pac-Man. The monkeys’ decision-making may be described with a strategy-based hierarchical decision-making model with over 90% accuracy. The model reveals that the monkeys adopted the take-the-best heuristic by using one dominating strategy for their decision-making at a time and formed compound strategies by assembling the basis strategies to handle particular game situations. With the model, the computationally complex but fully quantifiable Pac-Man behavior paradigm provides a new approach to understanding animals’ advanced cognition. One-Sentence Summary Macaque monkeys play Pac-Man with strategy-based hierarchical decision making, a cognitive capacity hitherto unknown in them.
BibTeX
@misc{yang2021monkey,
  author = {Qianli Yang and Zhongqiao Lin and Wenyi Zhang and Jianshu Li and Xiyuan Chen and Jiaqi Zhang and Tianming Yang},
  title = {{Monkey Plays Pac-Man with Compositional Strategies and Hierarchical Decision-making}},
  year = {2021},
  doi = {10.1101/2021.10.02.462713},
  howpublished = {bioRxiv},
}

Loading related content...

Figure