Hyperharmonic analysis for the study of high-order information-theoretic signals

Anibal M. Medina-Mardones, Fernando E. Rosas, Sebastián E. Rodríguez, Rodrigo CofréarXiv 2020 · arXiv

Introduction

Traditional network science has long relied on pairwise interactions—represented as edges between two nodes—to model complex systems. However, many real-world phenomena, from the synchronized firing of neurons in the human brain to the intricate dependencies of financial markets, exhibit "high-order" interactions. These are synergistic behaviors that emerge only when three or more components interact simultaneously, and they cannot be fully captured by decomposing the system into simple pairs.

While multivariate information theory provides tools like O-information to quantify these interactions, it faces a significant hurdle: the curse of dimensionality. As the number of variables ($N$) increases, the number of possible high-order combinations grows super-exponentially, making the resulting data nearly impossible to visualize or analyze. This paper introduces hyperharmonic analysis, a novel framework that combines combinatorial topology with information theory to find a compact, low-dimensional representation of these complex signals.

Theoretical Framework

High-Order Information Metrics

The authors utilize two primary metrics to characterize dependencies:

  1. O-information ($\Omega$): A signed metric where negative values indicate "synergy" (the whole provides more information than the sum of its parts) and positive values indicate "redundancy."
  2. S-information ($\Sigma$): A measure of total dependency within a group of variables.

Combinatorial Topology and the Simplicial Complex

To move beyond simple graphs, the system is modeled as a simplicial complex. In this structure, interactions are represented by simplices: nodes (0-simplices), edges (1-simplices), triangles (2-simplices), and tetrahedra (3-simplices). This geometric approach allows the researchers to treat information-theoretic metrics as "signals" residing on a specific topological shape.

The Discrete Laplace-de Rham Operator

The core innovation of this work is the application of the discrete $n$-Laplace operator ($\Delta_n$) to simplicial complexes. In classical signal processing, the Fourier transform uses the eigenfunctions of a Laplacian on a circle (sines and cosines) to decompose signals. Similarly, the authors diagonalize the discrete Laplace-de Rham operator to obtain a "hyperharmonic" basis. This basis is geometrically tuned to the underlying structure of the system, providing a natural coordinate system for high-order data.

Experimental Validation

Proof-of-Concept: Haydn’s London Symphonies

The authors applied their framework to electronic scores of Franz Joseph Haydn’s "London Symphonies" (Nos. 93–94 and 99–104). In this model:

  • Nodes: Individual instruments in the orchestra.
  • Signals: The informational dependencies (O-information) between instrument groups during performance.

Synthetic Benchmarks

To test the robustness of the compression, the authors also utilized synthetic multivariate Normal distributions with controlled covariance ranks. This allowed them to observe how the "diversity" of input data influences the compressibility of the hyperharmonic representation.

Key Insights and Results

Massive Dimensionality Reduction

The hyperharmonic representation demonstrated a remarkable ability to concentrate information. In the analysis of Haydn’s symphonies, the authors examined 4-dimensional signals (interactions between groups of five instruments). They found that just one component in the hyperharmonic basis could account for 60% of the variance. In contrast, the standard canonical basis required 35 components to achieve the same level of explanatory power. This efficiency remained consistent across various orders of interaction (from 3-way to 6-way groupings).

Geometric Localization

The results suggest that the geometry of the system—defined by its pairwise correlations—dictates the distribution of higher-order information. The hyperharmonic basis effectively captures the "modes" of interaction that are most relevant to the system's structure. Furthermore, the study indicates that while O-information and S-information both capture total dependency, they may be localized differently across the simplicial complex, suggesting they offer complementary views of the system's organization.

Superiority over Random Bases

To ensure the results were not a byproduct of any arbitrary basis change, the authors conducted control experiments using randomly generated bases. These random bases failed to achieve the compression levels of the hyperharmonic basis, proving that the Laplace-de Rham operator is uniquely suited to the system's intrinsic relational geometry.

Limitations and Future Directions

While hyperharmonic analysis offers a powerful tool for dimensionality reduction, the initial diagonalization of the Laplace matrix can be computationally intensive for extremely large systems. The authors suggest that "partial diagonalization" techniques could mitigate this in future applications.

Additionally, the current framework focuses on synchronous interactions (relationships occurring at the same time). Expanding the methodology to include "diachronous" or time-lagged relationships would be a significant step toward causal discovery in complex systems. Finally, while the mathematical efficiency of the hyperharmonic modes is clear, further domain-specific research is needed to interpret exactly what these modes represent in biological or musical contexts.

Conclusion

Hyperharmonic analysis bridges the gap between high-order information theory and combinatorial topology. By treating information as a signal on a simplicial complex, the framework provides a principled way to untangle the complexity of synergistic systems, offering a path forward for the study of high-dimensional data in fields ranging from neuroscience to musicology.

Summary updated

Abstract

Network representations often cannot fully account for the structural richness of complex systems spanning multiple levels of organisation. Recently proposed high-order information-theoretic signals are well-suited to capture synergistic phenomena that transcend pairwise interactions; however, the exponential-growth of their cardinality severely hinders their applicability. In this work, we combine methods from harmonic analysis and combinatorial topology to construct efficient representations of high-order information-theoretic signals. The core of our method is the diagonalisation of a discrete version of the Laplace-de Rham operator, that geometrically encodes structural properties of the system. We capitalise on these ideas by developing a complete workflow for the construction of hyperharmonic representations of high-order signals, which is applicable to a wide range of scenarios.
BibTeX
@misc{medinamardones2020hyperharmonic,
  author = {Anibal M. Medina-Mardones and Fernando E. Rosas and Sebastián E. Rodríguez and Rodrigo Cofré},
  title = {{Hyperharmonic analysis for the study of high-order information-theoretic signals}},
  year = {2020},
  eprint = {2010.01117},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.01117},
}
Figure