Introduction
Deep Neural Networks (DNNs) have become the standard for high-accuracy image classification due to their ability to learn complex features automatically. However, they are often criticized for being "black boxes"—computationally expensive and difficult to interpret. Conversely, Decision Trees (DTs) offer high interpretability and efficiency but struggle with high-dimensional data like raw images.
Self-born Wiring (SeBoW) is a novel framework designed to bridge this gap. It transforms standard DNNs into efficient, interpretable "neural trees." Unlike previous methods that grow trees greedily or use rigid structures, SeBoW utilizes a "construction-by-destruction" approach, allowing a "mother" network to autonomously rewire itself into a globally optimized tree structure.
Methodology: Construction-by-Destruction
The SeBoW framework follows a multi-stage process to derive a neural tree from a pre-existing high-performing architecture, such as VGG-13.
- Destruction (Unpacking): The mother network is partitioned into sections, and each layer is decomposed into isolated "filter groups" or base learners.
- Construction (Wiring): Two types of routers are introduced to manage data flow:
- Senders: Soft-max classifiers that determine which learners in the subsequent layer receive data.
- Receivers: Utilizing the Gumbel-Softmax trick, these components make discrete routing decisions differentiable, enabling the use of standard gradient descent for wiring optimization.
- Optimization: The process involves a search phase to identify the optimal wiring, followed by a retraining phase where the discovered tree structure is fine-tuned to maximize accuracy.
Experimental Evaluation
SeBoW was evaluated across several benchmarks, ranging from small-scale datasets to the massive ImageNet challenge.
- CIFAR-10 & CIFAR-100: Used for initial validation and comparison against baseline architectures.
- ImageNet: Serving as the gold standard, SeBoW's success on this dataset (1.33 million images) marks a significant milestone for neural tree research, which has historically struggled with scalability.
Key Insights and Semantic Hierarchy
A core advantage of SeBoW is its ability to learn a logical, semantic hierarchy. Rather than arbitrary splits, the model naturally learns to separate broad categories (e.g., animals vs. vehicles) at higher levels of the tree, refining these into specific classes (e.g., cars vs. trucks) at deeper levels.
This sparsity is not just a byproduct but a feature; the router design naturally prunes unnecessary connections, leading to a lightweight final model that offers both visual interpretability and computational efficiency.
Results and Performance
SeBoW consistently outperforms previous hybrid models and remains competitive with standard non-tree architectures:
- Efficiency: On CIFAR-10, SeBoW-C achieved 94.24% accuracy with only 4.6M parameters, significantly fewer than the 11.2M parameters required by a standard ResNet-18.
- Scalability: On ImageNet, SeBoW-C reached a 70.13% Top-1 accuracy, surpassing its mother network (VGG-13) while maintaining a more efficient structure.
- Inference Speed: The model supports a "single-path" inference mode, which executes only a fraction of the total neurons, providing substantial speedups over traditional dense networks.
Conclusion
SeBoW represents a significant advancement in the development of scalable and interpretable neural trees. By leveraging global differentiable optimization, it avoids the local optima common in greedy growth methods. While currently dependent on a pre-defined mother architecture, future research aims to expand this framework to more diverse tasks like object detection and segmentation.
Summary updated