Abstract
Deep Neural Networks (DNNs) are typically optimized for a fixed computational cost, which is problematic in real-world scenarios with dynamic time constraints. We present "Impatient DNNs," a framework that enables standard CNNs to provide a series of increasingly accurate predictions during a single forward pass. By incorporating early prediction layers and training them jointly using a budget-aware risk minimization strategy, these models achieve superior performance under time pressure. Our results on scene recognition benchmarks demonstrate significant accuracy gains and improved efficiency through uncertainty-based early stopping.
Introduction
In many practical applications, such as autonomous driving or robotics, the time available for a model to make a prediction is not constant. A system might need to react immediately to an unexpected obstacle or adjust its processing depth based on fluctuating energy resources. Traditional methods for speeding up DNNs, such as pruning or hardware acceleration, typically offer a static trade-off between speed and accuracy.
Impatient DNNs address this by being "anytime" models. They can be interrupted at various stages of computation, providing the best possible classification result for the time consumed so far.
Methodology
The core of the Impatient DNN framework involves modifying standard architectures (e.g., AlexNet, VGG19) to support multiple exit points.
Early Prediction (EP) Layers
Auxiliary "early prediction" branches are added at different depths of the network. These layers allow the model to output a classification result before reaching the final layer. Among various designs, the AVG 4x4 architecture—which utilizes spatial average pooling—was found to be most effective at reducing parameter overhead while maintaining high accuracy.
Joint Risk Minimization and Budget-Aware Weighting
Unlike training exits independently, Impatient DNNs use a joint training approach. The total loss is a weighted sum of losses from all exits. The weights for the different exits are derived from the expected distribution of time budgets (e.g., linear, equal, or polynomial distributions). This allows the model to prioritize accuracy at specific depths based on the application's requirements. Batch Normalization was found to be critical for the stability and convergence of this multi-branch training process.
Experimental Evaluation
The framework was evaluated on the 15-Scenes and MIT-67 benchmarks. Models pre-trained on ImageNet were fine-tuned using the Impatient framework.
Key Results
- Accuracy under Pressure: On MIT-67 with a VGG19 backbone, Impatient DNNs improved expected accuracy by nearly 10% for low-time budgets compared to standard baselines.
- Efficiency Gains: By implementing uncertainty-based stopping (comparing the top two class probabilities), an Impatient AlexNet achieved 57% accuracy (up from 53%) without increasing average computation time.
- Deep Supervision: Joint training often improved the accuracy of the final layer itself, suggesting that auxiliary losses provide beneficial regularization and help gradients flow better through deep networks.
Conclusion and Future Work
Impatient DNNs provide a robust solution for environments with unpredictable computational resources. While the early prediction layers introduce a minor parameter overhead, the benefits in flexibility and anytime accuracy are substantial. Future research could explore applying this framework to reinforcement learning, where rapid "early looks" could accelerate decision-making in time-critical tasks.
Summary updated