Abstract
Many real-world systems studied are governed by complex, nonlinear dynamics. By modeling these dynamics, we can gain insight into how these systems work, make predictions about how they will behave, and develop strategies for controlling them. While there are many methods for modeling nonlinear dynamical systems, existing techniques face a trade-off between offering interpretable descriptions and making accurate predictions. Here, we develop a class of models that aims to achieve both simultaneously, smoothly interpolating between simple descriptions and more complex, yet also more accurate models. Our probabilistic model achieves this multi-scale property through a hierarchy of locally linear dynamics that jointly approximate global nonlinear dynamics. We call it the tree-structured recurrent switching linear dynamical system (TrSLDS). To fit this model, we present a fully-Bayesian sampling procedure using Pólya-Gamma data augmentation to allow for fast and conjugate Gibbs sampling. Through a variety of synthetic and real examples, we show how these models outperform existing methods in both interpretability and predictive capability.
Introduction
Modeling complex, nonlinear systems—such as neural population activity or chaotic physical phenomena—often necessitates a compromise between interpretability and predictive accuracy. Simple linear models are easily understood but fail to capture complex behaviors, while high-capacity "black box" models like deep neural networks predict well but offer limited insight into underlying mechanisms.
Switching Linear Dynamical Systems (SLDS) attempt to bridge this gap by approximating global nonlinear dynamics as a sequence of locally linear regimes. However, traditional SLDS typically partition the state space sequentially, which fails to reflect the multi-scale nature of many real-world systems. To address this, we introduce the Tree-Structured Recurrent Switching Linear Dynamical System (TrSLDS), a model designed to provide both a coarse global view and a fine-grained local view of dynamics simultaneously.
The TrSLDS Framework
The core innovation of TrSLDS is the introduction of a hierarchical partitioning of the state space. Unlike standard recurrent SLDS (rSLDS) that use a flat list of regimes, TrSLDS employs a Tree-Structured Stick-Breaking process. To determine the active regime, the system traverses a tree from the root, making binary branching decisions at each node based on the current latent state. This creates a natural hierarchy of partitions where deeper nodes represent increasingly localized refinements of the state space.
To ensure the hierarchy is meaningful, we implement a Hierarchical Dynamics Prior. This prior enforces that "nearby" regimes in the tree behave similarly by centering a child node's linear dynamics parameters around those of its parent. Mathematically, this allows the model to be viewed as a "residual model," where the root captures global trends and children learn local corrections. Inference is made computationally feasible via Pólya-Gamma data augmentation, which enables fast, conjugate Gibbs sampling for the posterior distribution.
Experimental Results
We evaluated the TrSLDS on several benchmark systems, including the FitzHugh-Nagumo (FHN) oscillator, the Lorenz attractor, and real-world Macaque V1 neural recordings.
In the FHN oscillator example, the TrSLDS demonstrated its multi-scale utility. While the root node captured a general convergence toward a fixed point, the leaf nodes accurately reconstructed the complex limit cycle. This hierarchical decomposition allowed for superior $k$-step prediction performance ($R^2$) compared to standard SLDS and rSLDS models.
When applied to real neural data from the macaque primary visual cortex, the model successfully identified distinct limit cycles corresponding to different visual stimulus orientations. This provided a more structured and interpretable representation of neural population dynamics than previous state-of-the-art methods, effectively separating global stimulus-driven responses from local variations.
Conclusion
The TrSLDS offers a powerful framework for multi-scale modeling, balancing the need for interpretable global descriptions with the necessity of local accuracy. While the current implementation requires a pre-specified tree structure and is sensitive to initialization, it represents a significant step toward automated discovery of hierarchical structure in complex data. Future work will focus on learning the optimal tree topology directly from data and improving the robustness of the inference procedure.
Summary updated