Partial Information Decomposition for Complex Systems

This direction is merged into another direction: partial information decomposition. Its last version stays readable and citable here.

Version 1 of 127 papers33 conceptsfrozen 2026-10-04

Standard information theory, pioneered by Claude Shannon, tells us how much information is shared between two variables. However, it remains silent on how that information is distributed. If two sensors provide information about a target, do they provide the same information (redundancy), different pieces of information (uniqueness), or information that only emerges when the two are combined (synergy)?

Partial Information Decomposition (PID) is a theoretical framework designed to "crack open" mutual information and reveal these hidden components. By partitioning information into distinct "atoms," researchers like William A. Phillips, Joseph T. Lizier, and Viola Priesemann argue we can finally distinguish between simple information relay and complex neural processing in both biological and artificial systems Partial information decomposition as a unified approach.

The Underdetermined Puzzle of Information Atoms

The core challenge of PID is that it is mathematically "underdetermined." As Aaron J. Gutknecht, Abdullah Makkeh, and Michael Wibral explain in From Babel to Boole, we know the total mutual information (the "whole"), but we have more unknown "atoms" (redundancy, synergy, and unique components) than we have independent equations to solve for them.

To resolve this, researchers have proposed various mathematical axioms to define a "redundancy" measure, but these often conflict. In Novel Inconsistency Results for Partial Information Decomposition, Abdullah Makkeh and colleagues prove that several intuitively desirable properties for these atoms are mathematically incompatible. This forces a trade-off: researchers must choose which logical properties to prioritize based on the system they are studying.

From Static Information to Dynamic Flow

While basic PID looks at static snapshots of data, Integrated Information Decomposition ($\Phi$ID) extends these concepts to time-series and dynamical systems. In Beyond integrated information, Pedro A. M. Mediano and Fernando E. Rosas introduce a taxonomy of 16 distinct atoms of information flow.

This framework allows scientists to see the "gears" of a system turning. For example, it can distinguish between:

  • Information Storage: Information that a variable maintains over time.
  • Information Transfer: Information moving from one variable to another.
  • Synergistic Integration: Information that only exists in the collective dynamics of the whole system.

This has profound implications for neuroscience. Using $\Phi$ID, Alberto Liardi and Robin L. Carhart-Harris have shown that psychedelic states consistently shift brain dynamics toward higher synergy, suggesting a more "integrated" but less "redundant" mode of processing Null models for comparing information decomposition across complex systems.

Grounding Information in Logic and Geometry

To move away from arbitrary definitions, recent work has attempted to ground PID in formal logic and mereology (the study of parts and wholes). Aaron J. Gutknecht and Michael Wibral argue in Bits and pieces that information atoms correspond to a hierarchy of logical constraints. By viewing information through the lens of Boolean logic, researchers can use tools like Fourier analysis to interpret abstract concepts like "synergy" as specific types of sensitivity in a system's response Partial Information Decomposition of Boolean Functions.

Furthermore, Fernando E. Rosas and colleagues have introduced entropic conjugation, a principle that unifies the "zoo" of high-order information measures Characterising high-order interdependence via entropic conjugation. This mathematical symmetry helps identify which metrics, like O-information, are most efficient at balancing the tension between synergy and redundancy.

Scaling to Deep Learning and Neural Codes

As systems grow in complexity, traditional PID becomes computationally impossible due to the combinatorial explosion of atoms. To solve this, Aaron J. Gutknecht and Pedro A. M. Mediano proposed Shannon invariants Shannon invariants: A scalable approach to information decomposition. These global metrics capture the average degree of redundancy or vulnerability in a system, allowing for the analysis of high-order interactions in deep neural networks.

In biological systems, these tools are being used to decode the "language" of the brain. Giuseppe Pica and Christopher D. Harvey developed Intersection Information ($I_{II}$) to quantify how much sensory information is actually used to drive behavior, finding that precise spike timing is a key "behavioral code" Quantifying how much sensory information in a neural code is relevant for behavior. Similarly, Jan Bím and Andrea Brovelli introduced Feature-specific Information Transfer (FIT) to track the flow of specific sensory features across neural networks A Non-negative Measure Of Feature-Related Information Transfer.

By moving beyond "how much" information is present to "what kind" of information is being processed, Partial Information Decomposition provides a rigorous mathematical foundation for understanding the architecture of complexity.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_partial_information_decomposition_for_co_7bd093cf,
  title = {Partial Information Decomposition for Complex Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/partial-information-decomposition-for-complex-systems/rdr_7bd093cf728241cc9b9d3bcafe4803b5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_partial_information_decomposition_for_co_7bd093cf_v1,
  title = {Partial Information Decomposition for Complex Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/partial-information-decomposition-for-complex-systems/rdr_7bd093cf728241cc9b9d3bcafe4803b5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}