TL;DR
This paper introduces a novel "latent goal model" to understand and predict complex, dynamic strategic interactions, moving beyond simplified game theory paradigms. By modeling players' moment-to-moment intentions as evolving "goals," the approach accurately captures the rich, continuous behavior of monkeys playing a real-time video game, offering a powerful tool for studying social dynamics and potentially neural correlates of decision-making.
What Problem Does This Paper Address?
Traditional approaches to studying strategic decision-making in neuroscience often rely on simplified experimental setups. These typically involve discrete choices, static interactions, and well-defined optimal solutions (like Nash equilibria). However, real-world social interactions are far more complex: they involve continuous actions, ongoing dynamics, and often lack a clear optimal strategy. This disconnect makes it challenging to bridge findings from controlled lab experiments with naturalistic behavior. The authors aim to bridge this gap by developing a computational model that can capture the nuances of dynamic, strategic interactions in a more ecologically valid setting.
Methods
The core of the paper is a generative model that describes how players' actions (joystick movements) arise from underlying, unobserved "latent goals."
- Latent Goal Model: At each moment, each player has a desired onscreen location (a "goal"). These goals are not static but evolve dynamically based on the current game state (positions, velocities of both players) and a learned "value function" that guides the player towards desirable areas. This value function is parameterized by a neural network, allowing for flexible modeling of complex interactions.
- Control Model: A proportional-integral-derivative (PID) controller translates the latent goal into actual joystick commands, accounting for the physical constraints of the joystick.
- Generative Process: The model describes a probabilistic process where observed trajectories are generated from these evolving goals and control signals.
- Approximate Bayesian Inference: To infer the latent goals from observed trajectories, the authors employ a scalable Variational Bayes approach, using neural networks to approximate the posterior distribution of the goals.
- Gaussian Mixture Model (GMM): The value function is modeled as a mixture of Gaussians, allowing for multiple potential goals or "sub-strategies" at any given time.
Datasets
- Primary Dataset: Behavioral data from a custom-built "penalty shot" task involving two rhesus macaques playing against each other. This dataset consists of puck and goalie trajectories recorded over 120 behavioral sessions, totaling 59,884 trials.
- Training/Evaluation: For demonstrating model fits and generating synthetic data, a subset of 8659 trials from the last 10 sessions of the experiment was used.
Key Insights and Ideas
- Goals as Momentary Intentions: The model proposes that complex strategic behavior can be understood as the execution of dynamically evolving latent goals, offering a more intuitive and interpretable representation of intention than raw motor commands.
- Value Function Captures Strategic Interaction: The learned value function, parameterized by a neural network, implicitly captures the complex interplay between players, dictating how each player's goals are influenced by the opponent's actions. This allows for modeling strategies without explicitly modeling beliefs or theory of mind.
- Generative Power for Counterfactuals: The generative nature of the model allows for simulating "what-if" scenarios by manipulating task parameters or player goals, enabling controlled experiments that would be difficult to conduct otherwise.
- Strategic Complexity as a Predictor: The number of direction changes (change points) in a player's trajectory serves as a quantifiable measure of strategic complexity, which was found to correlate with win rates.
Key Results
- Accurate Model Fits: The model accurately predicts control signals and their derivatives, and generates synthetic data that captures the qualitative features of real player behavior (Fig 4).
- Goals Predict Outcomes: Inferred goal states were found to be more predictive of final puck position than instantaneous velocity or gaze position for a significant portion of the trial, suggesting they capture crucial strategic information (Fig 5).
- Model Superiority: The proposed GMM-based latent goal model outperformed simpler alternatives (single Gaussian or linear models) in fitting control signals, generating realistic trajectories, and completing trials (Fig 6).
- Complexity and Performance: Players adopted more complex strategies (more change points) over time, and this increased complexity was associated with higher win rates for both shooters and goalies (Fig 8).
Limitations and Future Work
- No Explicit Theory of Mind: The model does not explicitly model players' beliefs about their opponents' intentions or mental states. While the value function might implicitly capture some of this, teasing apart these constructs is a direction for future work.
- Censoring of Control Signal: The model does not fully account for the fact that joystick inputs are a truncated version of the desired control signal. A more natural model would address this censoring.
- Extension to Hierarchical Models: The current value function is a mixture of Gaussians. Future work could explore extensions using Hidden Markov Models to capture longer-term "sub-strategies."
- Generalizability: While the model is designed to be generalizable to other time series data, its performance on entirely novel or significantly different tasks remains to be fully explored.
- Neural Correlates: The authors suggest that the inferred latent variables (goals, strategic complexity) could serve as valuable correlates for neural activity, opening avenues for future neuroscientific investigations.
Summary updated