Abstract: We introduce graph gamma process (GGP) linear dynamical systems to model real-valued multivariate time series. GGP generates $S$ latent states that are shared by $K$ different communities, each of which is characterized by its own pattern of activation probabilities imposed on a $S\times S$ directed sparse graph, and allow both $S$ and $K$ to grow without bound. For temporal pattern discovery, the latent representation under the model is used to decompose the time series into a parsimonious set of multivariate sub-sequences generated by formed communities. In each sub-sequence, different data dimensions often share similar temporal patterns but may exhibit distinct magnitudes, and hence allowing the superposition of all sub-sequences to exhibit diverse behaviors at different data dimensions. On both synthetic and real-world time series, the proposed nonparametric Bayesian dynamic models, which are initialized at random, consistently exhibit good predictive performance in comparison to a variety of baseline models, revealing interpretable latent state transition patterns and decomposing the time series into distinctly behaved sub-sequences.
Introduction
Modeling complex, real-valued multivariate time series requires capturing both local linear dynamics and global nonlinear transitions. While Linear Dynamical Systems (LDS) are valued for their simplicity and interpretability, they often fail to represent nonlinear phenomena. Switching LDS (SLDS) attempts to bridge this gap by partitioning time series into segments governed by different linear models. However, traditional SLDS models suffer from rigid "hard" transitions, a lack of state sharing between regimes, and the need for manual parameter tuning regarding the number of states.
To address these challenges, we introduce the Graph Gamma Process Linear Dynamical System (GGP-LDS). This nonparametric Bayesian framework allows latent states to be shared across multiple "communities" of dynamics, enabling smooth transitions and highly interpretable data decompositions.
Methodology: The Graph Gamma Process
The core innovation of GGP-LDS is the use of a Graph Gamma Process (GGP) to govern the transition matrix of the underlying LDS. Instead of a single monolithic transition matrix, the model conceptualizes the transition structure as a directed sparse graph formed by the superposition of multiple latent communities.
Key technical components include:
- Overlapping Communities: The model views the transition structure as a graph formed by the "logical OR" of $K$ latent communities. Each community represents a specific dynamic behavior, such as a rotation or a linear trend.
- Nonparametric Complexity: By leveraging a Gamma Process, the model can accommodate an infinite number of latent states ($S$) and communities ($K$). It automatically "shrinks" unnecessary components toward zero, discovering the optimal complexity for the given dataset.
- Spike-and-Slab Sparsity: A binary mask ($Z$) is applied to the transition matrix, ensuring that relationships between latent states remain sparse, which enhances the interpretability of the learned dynamics.
Key Insights and Results
The GGP-LDS model shifts the paradigm from discrete switching to a compositional approach. Rather than assuming a system is in exactly one state, the model represents the current behavior as a blend of multiple communities. This allows for smoother modeling of physical processes and more nuanced decompositions.
Experimental Performance
The model was evaluated across three diverse domains:
- Lorenz Attractor: In approximating this chaotic system, GGP-LDS significantly outperformed standard LDS and state-of-the-art switching models (e.g., TrSLDS) in 10-step forecast accuracy. Notably, the model correctly identified that only 4 communities were necessary to represent the dynamics, despite being initialized with a larger budget.
- Pedestrian Trajectories: The model successfully decomposed complex 2D motion into meaningful sub-behaviors, such as straight paths and circular turns.
- Financial Data: When applied to stock prices, the model revealed shared market trends across different industries.
Conclusion
GGP-LDS provides a robust, nonparametric solution for multivariate time series analysis. By combining the interpretability of graph-based transitions with the flexibility of overlapping communities, it offers superior predictive performance and clear qualitative insights. While the current reliance on Gibbs sampling presents computational challenges for real-time applications, the model's ability to "reverse-engineer" hidden rules from data makes it a powerful tool for temporal pattern discovery.
Summary updated