Abstract
Deep neural networks (DNNs) are often viewed as "black boxes" because their internal logic is unpredictable and difficult to interpret. This research introduces the concept of "neural-path coding," inspired by biological neural circuits, to quantify and control how information flows through a network. By measuring the Euclidean distance (ED) between the neural-paths of different classes, the researchers provide a geometric explanation for model confusion: the shorter the distance between paths, the more likely the model is to make mistakes. Furthermore, the study proposes a "controlled learning process" using redirect layers to pre-assign neurons to specific classes, creating more interpretable architectures that maintain high accuracy.
Introduction
Despite the massive success of Deep Neural Networks, they remain notoriously opaque. In high-stakes fields like medicine or autonomous driving, the inability to explain why a network makes a specific decision is a significant barrier to trust. This paper addresses three fundamental questions: What does a network learn? Why is that knowledge unpredictable? And why are certain classes easily confused?
Inspired by neuroscience—where specific neural circuits in the brain encode specific decisions—the authors hypothesize that DNNs develop similar "neural-paths." By identifying and controlling these paths, we can move away from the black-box model toward a more "expectable" and transparent architecture.
The Neural-Path Hypothesis
The core idea is that information is transmitted between neurons along specific circuits to reach end neurons that encode a decision. In a standard training process, these paths often overlap, with neurons shared between multiple classes. This sharing is a primary source of unpredictability and error.
As shown in Figure 1, the "controlled learning process" aims to separate these paths, ensuring that specific neurons are dedicated to specific classes. This makes the internal logic of the network visible and predictable.
Methodology: Quantifying and Controlling Information Flow
The researchers developed a three-step technical framework to implement this hypothesis:
- Neural-Path Quantification: For a trained model, a "neural-path" is defined by identifying neurons or feature maps that show high average activation levels for a specific class. This allows the researchers to map the flow of information through hidden layers.
- Distance Metric: By defining a Euclidean Distance (ED) between the neural-paths of any two classes, the team can mathematically measure how similar the network's internal representations are.
- Redirect Layers: To build interpretability by design, the authors introduced "redirect layers." In these layers, specific feature maps are pre-assigned to specific classes. Custom loss functions ($loss_{pass}$ to encourage activation for the assigned class and $loss_{forbid}$ to suppress it for others) force the network to form distinct, class-specific lanes.
Results: The Geometry of Mistakes
Experiments conducted on MNIST, CIFAR-10/100, and ImageNet32 revealed a striking correlation: Mistakes are geometric. The inverse relationship between neural-path distance and error rate held true across various architectures, including MLPs, ResNets, and DenseNets.
A key finding, dubbed the "20/50 Rule," showed that the top 20% of class pairs with the smallest neural-path distances accounted for 50% to 60% of all incorrect predictions. This confirms that path proximity is a direct predictor of model confusion.
Crucially, the "controlled learning" approach proved that interpretability does not have to come at the cost of performance. In a "Dogs vs. Cats" experiment using a ResNet-50, the controlled model achieved 97.92% accuracy, nearly identical to the standard baseline of 98.02%. Visualization of these redirect layers provided a clear diagnostic tool: when the model failed, the "wrong" class-specific neurons were visibly firing.
Conclusion
By quantifying the internal circuits of a neural network, this research provides a path toward explainable AI. While scaling redirect layers to thousands of classes remains a challenge for future work, the ability to predict errors through geometric distance and steer the learning process marks a significant step toward making deep learning models truly understandable and trustworthy.
Summary updated