TL;DR
This paper introduces a novel "state-dependent drift diffusion model" (HMM-DDM) that allows decision-making parameters to change across trials, revealing that mice dynamically switch between "impulsive" and "deliberative" strategies. This work offers a more nuanced understanding of how animals navigate the speed-accuracy trade-off and opens doors for investigating the neural underpinnings of these adaptive behaviors.
What Problem Does This Paper Address?
Traditional drift diffusion models (DDMs) are powerful tools for understanding decision-making, but they often assume that the underlying decision-making process remains constant across trials. This is a significant simplification, as real-world behavior, especially in animals, often shows trial-to-trial variability in strategy. For instance, an animal might sometimes rush through a decision to get a reward quickly (impulsive) and at other times carefully consider the evidence to ensure accuracy (deliberative). This variability is crucial for understanding how animals adapt to changing environments and balance competing goals like speed and accuracy. The paper aims to bridge the gap between idealized normative models and the complex, dynamic reality of animal decision-making by accounting for these shifting strategies.
Methods
The core innovation is the state-dependent drift diffusion model (HMM-DDM). This model combines two established frameworks:
- Drift Diffusion Model (DDM): This is the standard model for describing how evidence is accumulated over time to reach a decision. It models the process as a noisy accumulation of evidence towards one of two boundaries, representing the choices. Key parameters include the drift rate (how quickly evidence favors one choice), the diffusion coefficient (noise), the boundary separation (how much evidence is needed), and the non-decision time (time before evidence accumulation starts).
- Hidden Markov Model (HMM): This model is used to capture the sequential, discrete nature of the latent states. The HMM assumes that the animal is in one of several discrete "states" at any given trial, and these states transition from one to another according to a probability matrix.
The HMM-DDM links these two by allowing the parameters of the DDM (e.g., drift rate, boundary separation) to be specific to each latent state identified by the HMM. This means that in one state, the DDM might have parameters reflecting fast, impulsive decisions, while in another state, it might have parameters for slow, deliberative decisions. The model is trained using an Expectation-Maximization (EM) algorithm to simultaneously infer the latent state sequence and the parameters of both the HMM and the DDMs associated with each state.
Datasets
The paper uses two main types of data:
- Synthetic Data: The model's parameter recovery capabilities were first tested on synthetic data generated from simulated HMM-DDMs with varying degrees of "separation" between the DDM parameters of different states. This allowed the researchers to assess how well their model could recover the true underlying parameters.
- Mouse Behavioral Data: The primary empirical dataset comes from three water-restricted mice performing a self-paced, two-alternative forced-choice (2AFC) task. This task involved responding to sequences of LED flashes to indicate a choice. The data included session, trial counts, accuracy, and average reaction times for each mouse.
Key Insights and Ideas
- Mice actively explore the speed-accuracy trade-off: The model reveals that mice don't just have a single strategy but dynamically switch between states that represent different points on the speed-accuracy continuum. This suggests a flexible and active exploration of how to best balance speed and accuracy depending on the trial or internal state.
- Beyond fixed strategies: The HMM-DDM moves beyond the assumption of fixed decision parameters per animal, allowing for trial-to-trial strategy shifts within a single session. This is crucial for understanding more realistic animal behavior.
- Rare, human-like deliberation: The model identified rare "high-bound states" where mice exhibited deliberation times and accuracies comparable to those observed in humans. This highlights the cognitive sophistication of rodent decision-making.
- Interpretable behavioral states: The identified latent states correspond to distinct behavioral signatures (e.g., long RTs/high accuracy vs. short RTs/low accuracy), making the model's findings interpretable in terms of behavioral strategies.
Key Results
- The HMM-DDM provided a better fit to mouse behavioral data than simpler one-state DDM models, as indicated by AIC and BIC criteria, and posterior predictive checks. A three-state model was found to be the best fit.
- The three identified states characterized distinct behavioral strategies:
- State 1: Long reaction times (RTs) and high accuracy (~80%), described as "deliberative, accuracy-oriented."
- State 2: Intermediate RTs and accuracy (~75%), described as "intermediate behavior."
- State 3: Very short RTs and low accuracy (~55%), described as "fast but error-prone."
- The model predicted that animals spent the most time in the highly inaccurate state, aligning with the overall lower accuracy observed in the data.
- Parameter recovery on synthetic data showed that the model could accurately estimate HMM and DDM parameters, especially with longer trial sequences and when the latent states were more distinct ("high separation").
Limitations and Future Work
- Outlier Trials: The model struggled with highly idiosyncratic trials (e.g., rapid, stimulus-independent choices) that didn't fit the DDM framework well. These were removed by applying specific criteria. Future work could extend the model to systematically address this "model mismatch."
- Non-decision Time Optimization: The non-decision time variable (τ) presented optimization challenges due to its non-smooth likelihood function. While a workaround was found, more elegant solutions are desired.
- Latent Variable Degeneracy: Like many latent variable models, the HMM-DDM can be degenerate, making it difficult to guarantee a globally optimal solution without careful initialization and optimization.
- Future Directions:
- Neural Mechanisms: Identifying the neural circuits underlying these identified decision states is a key future direction.
- Environmental Factors: Extending the model to investigate how external variables (reward history, uncertainty) influence state occupancy.
- Within-Trial Dynamics: Incorporating more detailed within-trial accumulation dynamics.
- Continuous States: Exploring continuous latent state models instead of discrete ones for potentially greater expressivity.
- Integrating other data: Incorporating neural recordings or keypoint data for richer observations.
Summary updated