Abstract
In this paper, we present a new approach to interpret deep learning models. By coupling mutual information with network science, we explore how information flows through feedforward networks. We show that efficiently approximating mutual information allows us to create an information measure that quantifies how much information flows between any two neurons of a deep learning model. To that end, we propose NIF, Neural Information Flow, a technique for codifying information flow that exposes deep learning model internals and provides feature attributions.
Introduction
Despite their success, deep neural networks (DNNs) remain "black boxes." Most current explainability (XAI) methods, like SHAP or Integrated Gradients, are model-agnostic—they look at how changing an input changes an output, but they often ignore the internal "plumbing" of the network itself. We argue that neglecting the topological structure of the model limits our understanding. We don't just need to know which features were important; we need to know how that information was processed, neuron by neuron, as it traveled from the input layer to the final prediction.
Neural Information Flow (NIF)
We introduce Neural Information Flow (NIF), which treats a DNN as a directed acyclic graph. The core components are:
- Mutual Information (MI) Estimation: We use MI to quantify the statistical dependence between neurons in consecutive layers. To handle individual data points (instance-wise explanations), we leverage Point-wise Mutual Information (PMI).
- Network Science Metrics: Once the "flow" between neurons is quantified, we apply classical graph theory concepts like Betweenness Centrality to identify "hub" neurons and Community Structure to group functional clusters.
- Feature Attribution: By calculating the product of NIF values along all possible paths from an input feature to an output class, we create a global attribution matrix.
Key Insights and Results
Our analysis reveals several critical aspects of DNN internals. For instance, ReLU activations effectively "shut off" certain pathways; in a trained model, many neurons contribute zero information flow for specific classes. By using Betweenness Centrality, we found that only a small subset of neurons in the final hidden layers are "central" to the model's decision-making.
Information-Theoretic Pruning
Because NIF identifies which edges carry the most information, it provides a natural roadmap for pruning. We demonstrated that we could zero out a significant number of weights based on NIF scores with minimal impact on accuracy, until reaching the theoretical limit of the class problem.
Performance Metrics
When compared against raw mutual information (the "ground truth" of statistical dependence), NIF achieved a Kolmogorov-Smirnov (K-S) statistic of 1.0, significantly outperforming SHAP (0.75) and Integrated Gradients (0.25). This suggests NIF's explanations are more mathematically aligned with the actual information content.
Conclusion and Future Work
While NIF provides deep insights, its computational complexity currently scales quadratically with layer dimensionality, posing challenges for massive architectures like Transformers. Future work will focus on optimization and exploring the theoretical links between NIF and VC-dimension to bound the hypothesis space of models. Applying NIF during training to guide architecture search or regularization also remains a promising direction.
Summary updated