Abstract
Over the past decade, Deep Convolutional Neural Networks (DCNNs) have shown remarkable performance in most computer vision tasks. These tasks traditionally use a fixed dataset, and the model, once trained, is deployed as is. Adding new information to such a model presents a challenge due to complex training issues, such as "catastrophic forgetting", and sensitivity to hyper-parameter tuning. However, in this modern world, data is constantly evolving, and our deep learning models are required to adapt to these changes. In this paper, we propose an adaptive hierarchical network structure composed of DCNNs that can grow and learn as new data becomes available. The network grows in a tree-like fashion to accommodate new classes of data, while preserving the ability to distinguish the previously trained classes. The network organizes the incrementally available data into feature-driven super-classes and improves upon existing hierarchical CNN models by adding the capability of self-growth. The proposed hierarchical model, when compared against fine-tuning a deep network, achieves significant reduction of training effort, while maintaining competitive accuracy on CIFAR-10 and CIFAR-100.
Introduction
Standard Deep Convolutional Neural Networks (DCNNs) are typically "static" learners, trained on fixed datasets and deployed without further modification. However, real-world data environments are dynamic, with new classes of information arriving incrementally. Attempting to train a pre-existing model on new classes without access to the original training data often leads to catastrophic forgetting, where new weight updates overwrite previously learned features, severely degrading performance on original tasks.
While retraining the entire model on a combined dataset is a common solution, it is computationally expensive and unsustainable as the number of classes grows. Tree-CNN addresses this by providing a "lifelong learning" framework—a model that grows and adapts to new information organically while preserving its existing knowledge base.
Architecture
The core of the proposed system is a hierarchical structure where each internal node contains a DCNN. This hierarchy consists of:
- Root Node: The top-level classifier that performs coarse classification, sorting input images into broad "super-classes."
- Branch Nodes: Intermediate nodes that perform finer classification within a super-class.
- Leaf Nodes: The final classification labels representing specific categories.
The Self-Growth Algorithm
Tree-CNN is distinguished by its ability to self-grow. When new classes are introduced, the system passes a small sample (~10%) of the new data through the existing root node to generate a Likelihood Matrix ($L$). Based on the softmax likelihood values, the algorithm determines the optimal structural update:
- Add to existing child: If the new class strongly resembles an existing branch, it is appended to that branch.
- Merge and Branch: If the new class resembles two existing nodes, those nodes are merged into a new branch, and the new class is added there.
- Horizontal Expansion: If the new class is visually unique, it is added as a new child node directly to the root.
This "divide-and-conquer" approach ensures that only the root node and the specific affected branches require retraining, significantly reducing the computational overhead compared to full network fine-tuning.
Experimental Results
The researchers evaluated Tree-CNN using standard computer vision benchmarks, CIFAR-10 and CIFAR-100.
- Efficiency: On CIFAR-10, Tree-CNN achieved 86.24% accuracy while requiring only 60% of the training effort compared to full network retraining.
- Performance: In incremental tests on CIFAR-100, Tree-CNN significantly outperformed other popular incremental learning methods. It yielded 10% higher accuracy than iCaRL and over 50% higher accuracy than "Learning without Forgetting" (LwF).
- Resilience: While all networks show some degradation as classes increase, Tree-CNN's accuracy curve remained shallower than simple fine-tuning methods, demonstrating superior resistance to catastrophic forgetting.
Conclusion and Future Work
Tree-CNN offers a scalable solution for lifelong learning by organizing data into feature-driven hierarchies. By localizing updates to specific branches, it maintains high accuracy with reduced training effort. Future work will address the increasing memory footprint as the network grows and explore the implications of tree depth on inference latency and memory requirements.
Summary updated