Toward a unified taxonomy of information dynamics via Integrated Information Decomposition

Pedro A. M. Mediano, Fernando E. Rosas, Andrea I. Luppi, Robin Carhart‐Harris, Daniel Bor, Anil K. Seth, Adam B. BarrettProceedings of the National Academy of Sciences 2025 · DOI

TL;DR

This paper introduces Integrated Information Decomposition (OID), a novel mathematical framework that breaks down complex system dynamics into fundamental "information atoms." OID allows us to disentangle and quantify various modes of information flow, transfer, storage, and integration, offering a more nuanced understanding of complex systems than previous methods. This framework has been validated across over 1,000 diverse datasets, leading to refined analyses of dynamical complexity and resolving theoretical ambiguities in existing measures.

What Problem Does This Paper Address?

Understanding and controlling complex systems – from the human brain to global economies – is a major scientific challenge. A key hurdle is our limited ability to describe and quantify the intricate, many-to-many interactions that drive their behavior. While various metrics exist to assess aspects like information transfer, storage, and overall complexity, they often conflate different underlying phenomena or are restricted in scope. For instance, Partial Information Decomposition (PID) can disentangle information from multiple sources about a single target, but it struggles with the ubiquitous many-to-many interactions found in real-world complex systems. This paper aims to bridge this gap by providing a unified framework that can handle these complex interactions and offer a more granular understanding of information dynamics.

Methods

The core contribution of this paper is the development of Integrated Information Decomposition (OID). OID extends the principles of PID to a "multitarget" scenario, allowing for the decomposition of information flow between multiple sources and multiple targets simultaneously. Specifically, OID focuses on how information atoms, previously defined by PID (redundant, unique, and synergistic), evolve over time. This temporal perspective leads to a richer taxonomy of 16 distinct "OID atoms," each representing a specific combination of past and future information dynamics (e.g., Red → Syn, Syn → Un¹).

To compute these OID atoms, the authors leverage a multitarget extension of existing information-theoretic measures, such as Barrett's Multivariate Mutual Information (MMI). The framework requires defining a "double-redundancy function" to resolve ambiguities, similar to how PID requires a redundancy function. The paper demonstrates that OID can be applied to various existing measures of integrated information and information transfer, revealing their underlying components and limitations.

Datasets

The authors validate their OID framework using a diverse collection of datasets:

  • Simulated Data:
    • Toy logic gate systems (copy, XOR, parity-preserving random) to illustrate theoretical concepts.
  • Simulated whole-brain activity from the Dynamic Mean Field (DMF) model.
  • Empirical Data:
    • A dataset of 1,053 multivariate time-series from biological, social, physical, and synthetic systems (from Cliff et al., 2023).
  • Heart rate and respiration data from the Fantasia database (40 healthy subjects).

Key Insights and Ideas

  • Beyond PID's Limitations: OID successfully extends information decomposition to handle many-to-many interactions, a crucial step for analyzing complex systems.
  • A Richer Taxonomy: The framework reveals 16 distinct "information atoms" that capture the temporal evolution of information flow, offering a more granular view than PID's three basic types.
  • Resolving Ambiguity in Existing Measures: OID demonstrates that popular measures like transfer entropy and integrated information are aggregates of these fundamental atoms, explaining why they can sometimes behave inconsistently or conflate different phenomena.
  • "Negatively Integrated" Systems Explained: OID provides a theoretical basis for why measures like "whole-minus-sum" (WMS) can yield negative values, attributing it to the overcounting of redundant information. A revised measure (ΦR) is proposed to correct this.
  • Information Duplication as a Common Denominator: The analysis of 1,053 datasets reveals that the correlation between active information storage (AIS) and transfer entropy (TE) is largely driven by a shared component: information duplication (Un¹ → Red). This suggests that these two measures might not be as independent as often assumed.
  • New Modes of Causation: OID highlights previously unreported modes of information dynamics, such as "upward causation" and "synergistic storage."

Key Results

  • Theoretical Dissection: OID successfully decomposes established measures like transfer entropy (TE) into four distinct OID atoms, showing that TE is not purely "transfer" but also includes synergistic and redundant components.
  • Integrated Information Measures Clarified: By analyzing how different integrated information measures (WMS, ΦG, Ψ, CD) respond to individual OID atoms, the study shows they capture distinct combinations of these atoms, explaining their varied behavior.
  • WMS Correction: The paper demonstrates that a revised measure (ΦR) that accounts for redundant information provides a more stable and interpretable measure of integration compared to WMS, especially in simulated whole-brain activity and noisy autoregressive systems.
  • Empirical Validation: Analysis of heart rate and respiration data shows that transfer entropy is driven by synergistic downward causation, not just pure transfer.
  • Large-Scale Dataset Analysis: Across 1,053 diverse datasets, OID reveals that the correlation between AIS and TE is significantly reduced when the shared component of information duplication is accounted for, suggesting a common underlying mechanism.

Limitations and Future Work

  • Redundancy Function Choice: OID, like PID, requires the specification of a "double-redundancy function" for numerical computation. The paper acknowledges that there isn't a universally agreed-upon choice, and empirical results might depend on this selection. Future work will involve exploring and comparing different redundancy functions and assessing the robustness of findings across them.
  • Non-Markovian Systems: The current framework primarily focuses on Markovian dynamics (where the future depends only on the present). The authors note that important phenomena in non-Markovian systems might be missed and suggest future work to address this, potentially using techniques like Taken's embedding theorem.
  • Focus on Two Time Points: The current formulation decomposes information at two time points. Future work could explore extensions to longer time series and systems with unobserved variables.
  • Mechanisms vs. Behavior: The current analysis focuses on observational data, describing system behavior. Future work could explore how OID can be used to investigate underlying causal mechanisms.

Summary updated

Abstract

Our ability to understand and control complex systems of many interacting parts remains limited. A key challenge is that we still do not know how best to describe-and quantify-the many-to-many dynamical interactions that characterize their complexity. To address this limitation, we introduce the mathematical framework of Integrated Information Decomposition, or [Formula: see text]ID. [Formula: see text]ID provides a comprehensive framework to disentangle and characterize the information dynamics of complex multivariate systems. On the theoretical side, [Formula: see text]ID reveals the existence of previously unreported modes of collective information flow, providing tools to express well-known measures of information transfer, information storage, and dynamical complexity as aggregates of these modes, thereby overcoming some of their known theoretical shortcomings. On the empirical side, we validate our theoretical results with computational models and examples from over 1,000 biological, social, physical, and synthetic dynamical systems. Altogether, [Formula: see text]ID improves our understanding of the behavior of widely used measures for characterizing complex systems across disciplines and leads to new more refined analyses of dynamical complexity.
BibTeX
@inproceedings{mediano2025toward,
  author = {Pedro A. M. Mediano and Fernando E. Rosas and Andrea I. Luppi and Robin Carhart‐Harris and Daniel Bor and Anil K. Seth and Adam B. Barrett},
  title = {{Toward a unified taxonomy of information dynamics via Integrated Information Decomposition}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2025},
  doi = {10.1073/pnas.2423297122},
}
Figure