Abstract
In this paper, we present a new approach to interpret deep learning models. By coupling mutual information with network science, we explore how information flows through feedforward networks. We show that efficiently approximating mutual information allows us to create an information measure that quantifies how much information flows between any two neurons of a deep learning model. To that end, we propose NIF, Neural Information Flow, a technique for codifying information flow that exposes deep learning model internals and provides feature attributions.
Introduction
Deep learning models are frequently criticized as "black boxes"—systems where inputs and outputs are visible, but the internal logic remains opaque. While existing interpretability tools like SHAP and LIME identify important features, they often treat the neural network as a model-agnostic function, ignoring the specific topology of the architecture.
Researchers from Carnegie Mellon, Cambridge, Arm, and UT Austin address this gap with Neural Information Flow (NIF). By treating a neural network as a Directed Acyclic Graph (DAG), NIF quantifies the statistical "flow" of information from input pixels to final classification. This approach allows for a deeper understanding of how specific neurons contribute to the model's decision-making process.
Methodology
NIF is built upon two theoretical pillars: Information Theory and Network Science.
Mutual Information (MI)
To quantify the statistical dependence between any two neurons, NIF utilizes Mutual Information. Because MI is traditionally difficult to compute in high dimensions, the authors employ advanced estimators (such as KSG and MINE) and a decomposition strategy that balances feature relevance against redundancy. This allows the framework to measure exactly how much information makes it from a specific input to a specific hidden neuron.
Network Science Metrics
Once the information flow between neurons is quantified, the network is modeled as a weighted graph. This enables the application of classical network metrics:
- Betweenness Centrality: Identifies "hub" neurons that serve as critical bridges for information.
- Community Structure: Uses modularity maximization to find clusters of neurons that work together to process specific types of information.
Key Insights and Results
Information-Theoretic Pruning
Unlike standard pruning techniques that rely on weight magnitude, NIF identifies weights that are informationally redundant. On benchmarks like the Banknote Authentication and Iris datasets, the authors demonstrated that a significant number of low-importance weights could be zeroed out while maintaining upwards of 96% accuracy. Accuracy only drops significantly once the "central" information-carrying nodes are removed.
Superior Attribution and Visualization
NIF provides instance-specific saliency maps (using Point-wise Mutual Information) that highlight which pixels provide the most information to internal channels. When compared to SHAP and Integrated Gradients, NIF's attributions were statistically closer to the ground-truth mutual information. In Kolmogorov-Smirnov (K-S) tests, NIF achieved a perfect distribution match (1.0), significantly outperforming SHAP (0.75) and Integrated Gradients (0.25).
Conclusion and Future Work
NIF offers a mathematically grounded framework for opening the neural network black box. It reveals that ReLU activations can act as "information dams" and that neurons naturally form functional "social" structures. However, the current implementation faces computational challenges, as the complexity scales quadratically with layer dimensionality. Future work will focus on optimizing NIF for large-scale architectures like Large Language Models (LLMs) and exploring theoretical links to model complexity measures like VC-dimension.
Summary updated