TL;DR
Researchers have developed WARHMM (Warped Autoregressive Hidden Markov Models), a new framework that disentangles what an animal is doing (discrete "syllables" like grooming or darting) from how they are doing it (continuous variability like speed or vigor). By adding a latent "warping" variable to standard behavioral models, WARHMM reduces the common problem of over-segmenting data and provides a precise way to quantify how factors like drugs or environment change the intensity of specific actions.
The Challenge: Capturing Continuous Variability in Discrete States
In systems neuroscience, a popular way to study behavior is to treat it like language: a sequence of stereotyped "syllables" (e.g., a rear, a left turn, a pause). The standard tool for this is the Autoregressive Hidden Markov Model (ARHMM), which segments video data into these discrete states.
However, animals aren't robots. A mouse might perform a "dart" very quickly or very slowly. Because standard ARHMMs only understand discrete states, they often mistake a "fast dart" and a "slow dart" for two entirely different behaviors. This leads to over-segmentation, where researchers end up with dozens of redundant behavioral clusters that are difficult to interpret. The gap in the field has been the lack of a model that can account for continuous, structured variability—like movement speed—within a single behavioral category.
Methods: The Warped ARHMM
To address this, the authors introduce the Warped ARHMM (WARHMM). It builds on the standard ARHMM but adds a continuous latent variable, $\tau_t$ (the "warping variable"), which modulates the dynamics of each syllable in real-time.
The paper explores two flavors of this warping:
- Time-Warped ARHMM (T-WARHMM): This version assumes that variability is primarily about speed. The warping variable acts like a "local clock," linearly scaling the dynamics. If $\tau$ is high, the animal moves through the behavioral trajectory faster; if it's low, the movement slows down.
- Gaussian Process-WARHMM (GP-WARHMM): This is a more flexible, non-linear version. It uses Gaussian Processes to allow the warping variable to change the dynamics in complex ways that aren't just about speed, though this comes at the cost of some interpretability.
To make the math tractable, the authors discretized the continuous warping variable onto a fine grid, allowing them to use efficient inference algorithms (like the forward-backward algorithm) typically used for standard HMMs.
Datasets and Validation
The model was validated using several datasets:
- Synthetic Data: 2D trajectories representing clockwise and counter-clockwise rotations at varying speeds tested if the model could correctly recover both the direction (discrete) and the speed (continuous).
- The MoSeq Dataset: Depth-camera recordings of 24 freely moving mice. The raw video was pre-processed into the top 10 principal components (PCs) of the mouse's posture.
- Pharmacological Dataset: Recordings of mice treated with either a saline solution (control) or amphetamine, used to see if the model could detect drug-induced changes in behavioral "vigor."
Key Insights and Results
- Vigor is a Latent Feature: Instead of just measuring how fast a mouse's center of mass moves (centroid velocity), WARHMM learns an internal "vigor" variable ($\tau$) that is specific to the geometry of the behavior being performed.
- Efficiency and Reduced Redundancy: A major advantage of WARHMM is that it requires fewer behavioral syllables to describe complex behavior. A WARHMM with only 25 syllables achieved the same performance (in terms of log-likelihood on held-out data) as a standard ARHMM with 40 syllables. In the mouse data, T-WARHMM halved the number of redundant "rear" states compared to the standard model, successfully collapsing fast and slow rears into a single, interpretable category.
- Drug Sensitivity: The model successfully identified that amphetamine-treated mice move with higher vigor. While a simple speed measurement showed a slight difference between groups, the T-WARHMM's $\tau$ variable showed a significant, syllable-specific shift toward higher speeds across almost all behaviors.
- Disentangling "What" from "How": The model proves that we can statistically separate the selection of an action from its execution. This is crucial for researchers studying the basal ganglia or dopamine, which are thought to regulate these two aspects of behavior separately.
Limitations and Future Work
While powerful, the current WARHMM framework has limitations. For computational ease, the authors turned the continuous warping variable into a discrete grid. A truly continuous inference method might provide even smoother results. Furthermore, the underlying model still assumes that behavior follows linear autoregressive dynamics; moving toward non-linear dynamics could capture even more nuance. Finally, the current model focuses primarily on speed. Future work could include other continuous axes of variability, such as the "radius" of a turn or the "height" of a posture, to create an even richer map of naturalistic behavior.
Summary updated