The Exponential Fragility of High-Order Feature Interactions

17 concepts9 papers1 proposal

When a deep neural network makes a prediction, it rarely relies on individual input features in isolation. Instead, it composes features into combinations—or interactions.

We can think of these interactions in terms of their order, which corresponds to the number of independent variables that must cooperate to form a concept. A low-order interaction might be a simple local texture or a single edge, requiring only a few nearby pixels to align. A high-order interaction represents a complex, global concept—such as the holistic shape or structural arrangement of an object—requiring the precise coordination of dozens or hundreds of variables across the entire input space.

Intuitively, high-order interactions sound like exactly what we want a sophisticated model to learn. They capture the rich, global structure of the world. However, a growing body of research reveals a fundamental mathematical trap: as the order of an interaction increases, its stability collapses exponentially.

The Mechanism of Exponential Noise Sensitivity

Why are high-order interactions so fragile? The core issue lies in how noise propagates through multi-variable dependencies.

Consider a logical "AND" relationship where $n$ distinct variables must all maintain their correct states for the interaction to fire. If even a single variable is perturbed, the entire relationship is invalidated. As theoretical work by Liu et al. and Zhou et al. demonstrates, the variance and inconsistency of an interaction effect increase exponentially relative to its complexity (or order) when subjected to input noise.

Qihan Ren et al. explain this phenomenon in Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Concepts: because high-order concepts require the precise alignment of multiple variables, a minor perturbation in just one constituent input can completely deactivate the representation. This extreme sensitivity means that high-order feature dependencies are inherently unstable across different samples, making them highly unreliable for generalization (Zhou et al.).

The Root of Adversarial Vulnerability

This mathematical instability directly explains why state-of-the-art deep neural networks are so notoriously brittle. When we witness adversarial examples—where an imperceptible buzz of noise completely flips a model's prediction—we are observing the collapse of high-order interactions.

In Discovering and Explaining the Representation Bottleneck of DNNS, Huiqi Deng shows that deep networks suffer from a representation bottleneck that biases them toward encoding either very simple or very complex interactions. While high-order interactions allow the model to capture global structural information, they simultaneously serve as the primary source of adversarial vulnerability.

Adversarial attacks exploit this exact asymmetry. Jie Ren et al. demonstrate in Towards a Unified Game-Theoretic View of Adversarial Perturbations and Robustness that adversarial perturbations specifically target and disrupt these complex, global collaborations, while leaving simple local collaborations relatively stable.

This insight is reinforced by Demyanenko et al., who connect a model's sensitivity to input noise directly to its vulnerability to adversarial examples. When a model relies on high interaction orders, a tiny, adversarial shift in the input space triggers a massive, chaotic swing in the network's output. Furthermore, Jie Ren et al. show in Extracting Robust On-Manifold Interactions Encoded by Neural Networks that these high-order interactions are disproportionately affected by input perturbations, making it difficult to extract stable, robust explanations of network behavior.

How Networks Manage High-Order Fragility

Given that high-order interactions are so mathematically unstable, how do neural networks manage to generalize at all? The answer lies in the internal dynamics of training and regularization.

Layerwise Purging

In Layerwise Change of Knowledge in Neural Networks, Xu Cheng et al. track the evolution of these interactions layer by layer. They discover that deep networks initially overfit a massive variety of complex, high-order interactions in their early layers. However, as the representations progress through the network, later layers actively "forget" these brittle, redundant interactions, distilling the knowledge down to stable, generalizable low-order primitives.

Adversarial Training

We can actively steer networks away from this fragility. Jie Ren et al. show that adversarial training builds robustness precisely by forcing the model to abandon its reliance on complex, global collaborations and instead depend on simpler, category-specific local features. While Mingjie Li et al. note in Does a Neural Network Really Encode Symbolic Concepts? that adversarial training helps networks converge on human-aligned symbolic logic, high-order interactions remain fundamentally more vulnerable to noise than low-order ones.

Bayesian Uncertainty

Another elegant solution is introducing weight uncertainty. Qihan Ren et al. demonstrate that Bayesian Neural Networks (BNNs) are naturally biased against learning complex patterns. Because BNNs represent weights as probability distributions rather than fixed points, the continuous slight fluctuations act as a filter. This uncertainty destabilizes the fragile, high-order interactions that require precise alignment, preventing the network from relying on brittle relationships and forcing it to favor robust, low-order concepts instead.

Rethinking the Architecture

Is the fragility of high-order interactions an inescapable law of deep learning? Not necessarily. The vulnerability appears closely tied to standard architectures like standard rectified linear unit (ReLU) networks.

In contrast, Dmitry Krotov et al. demonstrate in Dense Associative Memory is Robust to Adversarial Inputs that models explicitly designed with higher-order interaction vertices can exhibit significant inherent resistance to adversarial perturbations. By fundamentally altering the mathematical structure of how features interact, these dense associative memory models can correctly classify adversarial examples that typically cause high-confidence failures in standard lower-order networks. This suggests that while high-order interactions are inherently fragile in standard deep architectures, alternative paradigms can restructure these interactions to achieve both global expressivity and structural robustness.

Key Foundational Work

Towards a Unified Game-Theoretic View of Adversarial Perturbations and Robustness (2021) — contributed 1 concept elements to this direction.
Dense Associative Memory is Robust to Adversarial Inputs (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_interaction_stability_10c6c35d,
  title = {interaction stability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/interaction-stability/rdr_10c6c35d81514e76ad08932a09f1f381},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_interaction_stability_10c6c35d_v1,
  title = {interaction stability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/interaction-stability/rdr_10c6c35d81514e76ad08932a09f1f381/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake