TL;DR
This paper introduces the Langevin Graph Network Approach (LaGNA), a novel method for inferring the underlying stochastic differential equations (SDEs) that govern complex, networked systems directly from observational data. LaGNA successfully disentangles self-dynamics, interaction dynamics, and stochastic diffusion, leading to interpretable models that outperform existing methods on both simulated and real-world datasets, including bird flocking and tau pathology spread in brains.
What Problem Does This Paper Address?
Many natural and engineered systems, from biological networks to social dynamics, are inherently complex and exhibit stochastic behavior. These systems are often best described by stochastic differential equations (SDEs), which capture both deterministic trends and random fluctuations. However, inferring these SDEs from observed data is a significant challenge. Traditional approaches often rely on predefined model structures or simplified assumptions about system dynamics, which may not accurately reflect real-world complexity. While recent advances in machine learning have enabled the inference of deterministic dynamics (ODEs), effectively learning the hidden SDEs of stochastic systems, especially those with network structures, remains an open problem. This paper aims to bridge this gap by developing a method that can learn interpretable SDEs from observational data, even when the underlying dynamics are unknown.
Methods
The core of the proposed method is the Langevin Graph Network Approach (LaGNA). LaGNA operates in two stages:
- Learning the Dynamics: In the first stage, LaGNA employs a graph neural network-inspired message-passing mechanism. This mechanism is designed to disentangle the contributions of three key dynamic sources: self-dynamics (how a node evolves on its own), interaction dynamics (how nodes influence each other), and intrinsic stochastic diffusion (random noise affecting each node). Three specialized neural network modules (f, g, and φ) are trained to capture these respective components. The network is trained by maximizing the likelihood of observing the next state given the current state and network topology.
- Interpretable Equation Inference: The second stage focuses on making the learned dynamics interpretable. By "penetrating" the trained neural network modules, LaGNA derives concise mathematical expressions for the self, interaction, and diffusion components. This is achieved by using pre-constructed libraries of elementary mathematical terms and employing a symbolic regression-like approach to identify the optimal combination of terms that best represent the learned dynamics. This process results in a final, interpretable SDE that describes the system's behavior.
LaGNA is designed to handle signed and weighted networks, which are common in biological and physical systems. It also incorporates a second-order extension for systems exhibiting more complex dynamics, such as flocking.
Datasets
The paper utilizes the following datasets:
- Simulated Data:
- Stochastic Hindmarsh-Rose (HR) neuronal dynamics on signed and weighted networks.
- Stochastic Rössler dynamics on weighted networks.
- Stochastic Lorenz networked system.
- Simulated second-order Vicsek model dynamics.
- Experimental Data:
- Bird Flocking: Homing pigeon flight data collected using GPS devices. This dataset consists of four sets of homing flights, with varying numbers of pigeons per set.
- Tau Pathology Diffusion: Experimental data from non-transgenic (NTG) mice injected with PHF tau. Pathology was measured at different time points (1, 3, 6, and 9 months post-injection) across various brain regions. Data was also collected from mice with the LRRK2G2019S mutation.
Key Insights and Ideas
- Decoupling Dynamics is Key: The success of LaGNA hinges on its ability to effectively separate the self, interaction, and diffusion components of a system's dynamics. This disentanglement is crucial for learning interpretable SDEs.
- Vicsek Model as More Than a Toy: The inference of a second-order Vicsek-like SDE from real bird flocking data provides strong evidence that the Vicsek model is not just a simplified theoretical construct but captures genuine flocking dynamics.
- Neuroanatomical and Spatial Influence in Tau Pathology: The inferred SDE for tau pathology diffusion highlights the significant role of both neuroanatomical connections (retrograde and anterograde) and spatial proximity in disease spread. This suggests that understanding these combined factors is vital for predicting and potentially intervening in tau pathology.
- Mutant Mice Exhibit Directional Tau Spread: The study reveals that tau pathology diffusion in mice with the LRRK2G2019S mutation shows a pronounced directional preference (retrograde) compared to NTG mice, offering new insights into the disease mechanisms.
- Interpretability for Scientific Discovery: By generating interpretable SDEs, LaGNA moves beyond mere prediction and offers a pathway for scientific discovery, allowing researchers to understand the underlying mechanisms of complex systems.
Key Results
- Superior Performance: LaGNA outperformed five state-of-the-art methods in inferring the dynamics of the stochastic Lorenz networked system, achieving errors two orders of magnitude lower.
- Flocking Dynamics: The inferred SDE for bird flocks closely resembled the second-order Vicsek model, validating the model's applicability to real-world phenomena. The learned SDE could accurately predict flocking trajectories and force fields.
- Tau Pathology Prediction: The inferred SDE for tau pathology diffusion accurately predicted the spread of pathology in mouse brains at different time points. It also demonstrated the significant influence of regional heterogeneity and outperformed a degenerate model.
- Mutant Mouse Dynamics: The study quantified the distinct retrograde preference in tau diffusion in LRRK2G2019S mutant mice compared to NTG mice.
Limitations and Future Work
- Inaccessible Node Activity: LaGNA currently assumes that the activity of all nodes is observable. Future work could explore inferring dynamics when some node activities are inaccessible, potentially by determining minimal sub-network structures.
- Extrinsic Noise: The current method treats all noise as intrinsic. While LaGNA shows some robustness to extrinsic noise, more pronounced levels may require denoising preprocessing steps. Future research should focus on better addressing extrinsic noise.
- Network Topology Inference: The current framework infers dynamics given a known network topology. Future work could aim to simultaneously infer both network topology and system dynamics, which is a more challenging task.
- Higher-Order Interactions: LaGNA could be extended to accommodate higher-order interactions (beyond pairwise) by incorporating additional terms into the interaction dynamics, though this would increase complexity.
- Data Requirements: While LaGNA requires only one trial of activity sequence and snapshots (not continuous time series), further exploration into minimal data requirements for complex systems would be beneficial.
Summary updated