Abstract
Understanding the internal cognitive states of animals remains a significant challenge in neuroscience, as behavior is often driven by latent beliefs rather than direct sensory-motor mappings. This paper presents a data-driven framework for extracting these internal belief dynamics directly from observable behavior using a switching semi-Markov jump process (SMJP). By modeling behavior in continuous time and accounting for action-dependent state transitions, the framework successfully identifies hidden cognitive states, such as "expectant waiting" in foraging macaques. This approach provides a robust bridge between outward actions and internal neural planning without requiring pre-defined behavioral templates.
Introduction
In the study of animal behavior, researchers can readily observe actions but lack direct access to the underlying thoughts that drive them. Animal behavior is rarely a simple reflexive response; it is guided by internal latent states—beliefs about the environment, expectations of future rewards, and long-term strategic plans.
Traditional methods for inferring these states often suffer from two primary issues. First, they frequently rely on "model-mismatch," where researchers assume an animal follows a specific mathematical model (e.g., a standard Reinforcement Learning agent) that may not accurately represent the animal's actual cognitive process. Second, they often lack temporal accuracy; models like Hidden Markov Models (HMMs) assume inter-action times follow an exponential distribution, failing to capture the complex internal timing and non-random pauses characteristic of real animal behavior.
Methods: The Switching Semi-Markov Jump Process
To address these limitations, the authors developed the switching semi-Markov jump process (SMJP). This framework operates in continuous time, allowing for the precise modeling of intervals between behavioral events.
The SMJP introduces several key innovations:
- Action-Dependent Transitions: The model treats actions as operators that can change the "generator" of internal states. For instance, an action like "moving" can switch the agent into a different set of internal belief dynamics compared to "staying."
- Uniformization: To handle the mathematical complexity of continuous-time jumps, the authors employed a uniformization technique. This maps the continuous process onto a sequence of discrete-time matrices, making the computation tractable.
- Learning via EM: An Expectation-Maximization (EM) algorithm is used to learn transition and emission patterns from raw behavioral data, ensuring the model remains data-driven.
Experimental Validation
The model was validated across three datasets of increasing complexity:
- Simulated Toy Data: Used to verify the SMJP's ability to recover known ground-truth parameters in a controlled 5-state environment.
- Optimal Agent Data: The SMJP was applied to a simulated "ideal" foraging agent (a POMDP). The model successfully recovered the agent's internal belief states, demonstrating "cognitive compression" by explaining behavior using the minimum necessary number of latent states.
- Macaque Foraging Data: Real-world data from monkeys in a two-box foraging task. The model analyzed lever presses, reward timings, and physical locations to identify structured internal states.
Key Insights and Results
The SMJP revealed that roughly 10–12 latent states were sufficient to describe complex macaque foraging behavior. A significant discovery was the identification of "expectant waiting" states—structured, self-exciting delay networks that reflect the monkey's internal timing rather than random pauses.
Furthermore, the model captured "guarding behavior," where monkeys remained near a box to protect a potential reward. These behaviors were associated with specific low-entropy latent states, indicating high certainty in the animal's internal model. By treating actions as operators, the researchers identified persistent subspaces in the monkey's cognitive architecture related to reward expectation.
Discussion and Future Work
While this framework successfully extracts belief dynamics from behavior, the ultimate goal is to link these latent states to specific neural activity. The extracted states can serve as regressors for simultaneous neural recordings, potentially identifying the brain circuits responsible for cognitive processes like "expectant waiting." Future iterations may move beyond discrete latent states toward fully continuous latent spaces to better capture the nuances of biological cognition.
Summary updated