Simple physical systems as a reference for multivariate information dynamics

Alberto Liardi, Madalina Sas, George Blackburne, William Knottenbelt, Pedro A. M. Mediano, Henrik Jeldtoft Jensen

Introduction

Information-theoretic measures have become indispensable tools for analyzing complex systems, from neural networks to global financial markets. These metrics, such as synergy and integrated information, aim to quantify how the interactions between parts give rise to a whole that is "more than the sum of its parts." However, a fundamental challenge remains: these measures typically rely on observed statistical distributions, which may not faithfully reflect the underlying physical mechanisms. This discrepancy creates a "ghost of causality," where statistical correlations are mistaken for dynamical interactions.

To address this, this research utilizes a "mathematical laboratory"—a simple, analytically tractable system of interacting random walkers—to ground abstract information metrics in known physical reality. By comparing information-theoretic results against the ground-truth mechanics of the system, the study identifies critical conditions under which these metrics succeed or fail to represent physical interactions.

The Interacting Random Walker Model

The authors employ a one-dimensional chain of $N$ interacting random walkers (RWs) as their primary model. Each walker's movement is governed by an Ornstein-Uhlenbeck (OU) process, which includes a stochastic component (noise) and a restoring force pulling the walker toward a central point. Crucially, the walkers are coupled to their neighbors via springs with an interaction strength $\gamma$.

This setup allows for the analytical derivation of several high-order information metrics, including:

  • Transfer Entropy (TE): Measuring directed information flow between walkers.
  • Partial Information Decomposition (PID): Breaking down information into redundant, unique, and synergistic components.
  • Integrated Information ($\Phi$): Quantifying the degree of system unification.
  • Causal Emergence Metrics: Assessing whether macroscopic features, such as the center of mass, exhibit autonomous dynamics.

The study solves the system exactly for $N=2$ and applies a first-order perturbative expansion for larger systems ($N > 2$). To validate these analytical findings, numerical simulations were conducted with up to 512 walkers across 10,000 ensembles.

The "Steady-State Trap" and Causal Intervention

One of the paper's most significant findings is the "steady-state trap." In many applications, researchers calculate information metrics based on the system's steady-state behavior. However, the authors demonstrate that in their model, synergy—information available only when considering multiple parts together—often appears to decrease as the physical interaction strength $\gamma$ increases.

This counter-intuitive result occurs because stronger physical interactions increase the correlation (redundancy) between walkers. At a steady state, this redundancy "masks" the dynamical synergy. To overcome this, the authors introduce causal intervention using the "do-calculus" framework. By resetting the system to a state of maximum entropy before observing its evolution, they isolate the system's dynamics from its prior statistical correlations. Under intervention, the results align with physical intuition: synergy increases as the interaction strength $\gamma$ increases.

Statistical Autonomy vs. Mechanical Emergence

The paper also clarifies the concept of "emergence." Metrics designed to detect causal emergence often flag the center of mass (c.o.m.) of the walkers as an emergent feature. However, the authors show that the c.o.m. appears "causally decoupled" even when the walkers do not interact at all ($\gamma = 0$).

This phenomenon is labeled "statistical autonomy." The c.o.m. follows a predictable path due to the law of large numbers, regardless of the individual parts' interactions. This suggests that many metrics currently used to identify "emergence" in complex systems may actually be detecting statistical autonomy rather than true mechanical emergence.

Key Takeaways for Information Dynamics

The study highlights several requirements for the meaningful application of information theory to physical systems:

  1. Timescales are Critical: Information measures align best with physical parameters when calculated over short timescales that match the system's intrinsic dynamics. Over longer periods, the "memory" of interactions is often lost to noise.
  2. Normalization is Essential: Because the total information in a system changes with interaction strength, raw PID values can be misleading. Normalizing these values (e.g., by joint mutual information) is necessary for accurate interpretation.
  3. The Danger of Mixing Scales: When calculating integrated information ($\Phi$) between microscopic components and macroscopic features, the metric can produce paradoxical results, such as showing higher integration for weaker interactions.

Limitations and Future Outlook

While providing a rigorous foundation, the study is limited by its assumption of weak coupling ($\mathcal{O}(\gamma)$) for large systems and its reliance on linear, Gaussian processes. Real-world biological or social systems are frequently non-linear and non-Gaussian, which may introduce further complexities.

Future research should apply the "causal intervention" approach to empirical data, such as EEG or financial time series, to distinguish between true dynamical emergence and simple statistical autonomy. There is a clear need for new information-theoretic measures that are sensitive to high-order mechanisms rather than just autonomous statistical behavior.

Summary updated

Abstract

Understanding a complex system entails capturing the non-trivial collective phenomena that arise from interactions between its different parts. Information theory is a flexible and robust framework to study such behaviours, with several measures designed to quantify and characterise the interdependencies among the system's components. However, since these estimators rely on the statistical distributions of observed quantities, it is crucial to examine the relationships between information-theoretic measures and the system's underlying mechanistic structure. To this end, here we present an information-theoretic analytical investigation of an elementary system of interactive random walkers subject to Gaussian noise. Focusing on partial information decomposition, causal emergence, and integrated information, our results help us develop some intuitions on their relationship with the physical parameters of the system. For instance, we observe that uncoupled systems can exhibit emergent properties, in a way that we suggest may be better described as ''statistically autonomous''. Overall, we observe that in this simple scenario information measures align more reliably with the system's mechanistic properties when calculated at the level of microscopic components, rather than their coarse-grained counterparts, and over timescales comparable with the system's intrinsic dynamics. Moreover, we show that approaches that separate the contributions of the system's dynamics and steady-state distribution (e.g. via causal perturbations) may help strengthen the interpretation of information-theoretic analyses.
BibTeX
@misc{liardi2025simple,
  author = {Alberto Liardi and Madalina Sas and George Blackburne and William Knottenbelt and Pedro A. M. Mediano and Henrik Jeldtoft Jensen},
  title = {{Simple physical systems as a reference for multivariate information dynamics}},
  year = {2025},
  eprint = {2504.10372},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.10372},
}
Figure