Tree-Structured Recurrent Switching Linear Dynamical Systems for Multi-Scale Modeling

Josue Nassar, Scott W. Linderman, Mónica F. Bugallo, Il Memming Park

Abstract

Many real-world systems studied are governed by complex, nonlinear dynamics. By modeling these dynamics, we can gain insight into how these systems work, make predictions about how they will behave, and develop strategies for controlling them. While there are many methods for modeling nonlinear dynamical systems, existing techniques face a trade-off between offering interpretable descriptions and making accurate predictions. Here, we develop a class of models that aims to achieve both simultaneously, smoothly interpolating between simple descriptions and more complex, yet also more accurate models. Our probabilistic model achieves this multi-scale property through a hierarchy of locally linear dynamics that jointly approximate global nonlinear dynamics. We call it the tree-structured recurrent switching linear dynamical system (TrSLDS). To fit this model, we present a fully-Bayesian sampling procedure using Pólya-Gamma data augmentation to allow for fast and conjugate Gibbs sampling. Through a variety of synthetic and real examples, we show how these models outperform existing methods in both interpretability and predictive capability.

Introduction

Modeling complex, nonlinear systems—such as neural population activity or chaotic physical phenomena—often necessitates a compromise between interpretability and predictive accuracy. Simple linear models are easily understood but fail to capture complex behaviors, while high-capacity "black box" models like deep neural networks predict well but offer limited insight into underlying mechanisms.

Switching Linear Dynamical Systems (SLDS) attempt to bridge this gap by approximating global nonlinear dynamics as a sequence of locally linear regimes. However, traditional SLDS typically partition the state space sequentially, which fails to reflect the multi-scale nature of many real-world systems. To address this, we introduce the Tree-Structured Recurrent Switching Linear Dynamical System (TrSLDS), a model designed to provide both a coarse global view and a fine-grained local view of dynamics simultaneously.

The TrSLDS Framework

The core innovation of TrSLDS is the introduction of a hierarchical partitioning of the state space. Unlike standard recurrent SLDS (rSLDS) that use a flat list of regimes, TrSLDS employs a Tree-Structured Stick-Breaking process. To determine the active regime, the system traverses a tree from the root, making binary branching decisions at each node based on the current latent state. This creates a natural hierarchy of partitions where deeper nodes represent increasingly localized refinements of the state space.

Comparison of sequential vs. tree-structured stick-breaking for state probability allocation.

To ensure the hierarchy is meaningful, we implement a Hierarchical Dynamics Prior. This prior enforces that "nearby" regimes in the tree behave similarly by centering a child node's linear dynamics parameters around those of its parent. Mathematically, this allows the model to be viewed as a "residual model," where the root captures global trends and children learn local corrections. Inference is made computationally feasible via Pólya-Gamma data augmentation, which enables fast, conjugate Gibbs sampling for the posterior distribution.

Experimental Results

We evaluated the TrSLDS on several benchmark systems, including the FitzHugh-Nagumo (FHN) oscillator, the Lorenz attractor, and real-world Macaque V1 neural recordings.

In the FHN oscillator example, the TrSLDS demonstrated its multi-scale utility. While the root node captured a general convergence toward a fixed point, the leaf nodes accurately reconstructed the complex limit cycle. This hierarchical decomposition allowed for superior $k$-step prediction performance ($R^2$) compared to standard SLDS and rSLDS models.

Results on the FitzHugh-Nagumo oscillator showing how different levels of the tree capture varying levels of dynamical detail and prediction performance.

When applied to real neural data from the macaque primary visual cortex, the model successfully identified distinct limit cycles corresponding to different visual stimulus orientations. This provided a more structured and interpretable representation of neural population dynamics than previous state-of-the-art methods, effectively separating global stimulus-driven responses from local variations.

Conclusion

The TrSLDS offers a powerful framework for multi-scale modeling, balancing the need for interpretable global descriptions with the necessity of local accuracy. While the current implementation requires a pre-specified tree structure and is sensitive to initialization, it represents a significant step toward automated discovery of hierarchical structure in complex data. Future work will focus on learning the optimal tree topology directly from data and improving the robustness of the inference procedure.

Summary updated

Abstract

Many real-world systems studied are governed by complex, nonlinear dynamics. By modeling these dynamics, we can gain insight into how these systems work, make predictions about how they will behave, and develop strategies for controlling them. While there are many methods for modeling nonlinear dynamical systems, existing techniques face a trade off between offering interpretable descriptions and making accurate predictions. Here, we develop a class of models that aims to achieve both simultaneously, smoothly interpolating between simple descriptions and more complex, yet also more accurate models. Our probabilistic model achieves this multi-scale property through a hierarchy of locally linear dynamics that jointly approximate global nonlinear dynamics. We call it the tree-structured recurrent switching linear dynamical system. To fit this model, we present a fully-Bayesian sampling procedure using Polya-Gamma data augmentation to allow for fast and conjugate Gibbs sampling. Through a variety of synthetic and real examples, we show how these models outperform existing methods in both interpretability and predictive capability.
BibTeX
@inproceedings{nassar2019tree,
  author = {Josue Nassar and Scott W. Linderman and Mónica F. Bugallo and Il Memming Park},
  title = {{Tree-Structured Recurrent Switching Linear Dynamical Systems for Multi-Scale Modeling}},
  booktitle = {International Conference on Learning Representations},
  year = {2019},
  url = {https://openreview.net/forum?id=UIeyVC_lP4P},
}
Figure