Abstract
Modern deep neural networks (DNNs) are achieving state-of-the-art results due to their capability to learn a faithful representation of the data they are trained on. In this chapter, we address two insufficiencies of DNNs, namely, the lack of robustness to corruptions in the data, and the lack of real-time deployment capabilities, that need to be addressed to enable their safe and efficient deployment in real-time environments. We introduce hybrid corruption-robustness focused compression (HCRC), an approach that jointly optimizes a neural network for achieving network compression along with improvement in corruption robustness, such as noise and blurring artifacts that are commonly observed. For this study, we primarily consider the task of semantic segmentation for automated driving and focus on the interactions between robustness and compression of the network. HCRC improves the robustness of the network by 8.39% absolute mean performance under corruption (mPC) on the Cityscapes dataset, and by 2.93% absolute mPC on the Sim KI-A dataset, while generalizing even to augmentations not seen by the network in the training process. This is achieved with only minor degradations on undisturbed data. Our approach is evaluated over two strong compression ratios (30% and 50%) and consistently outperforms all considered baseline approaches. Additionally, we perform extensive ablation studies to further leverage and extend existing state-of-the-art methods.
The Deployment Gap in Deep Learning
Deep learning models currently face a significant "deployment gap." While state-of-the-art models excel in accuracy, they are often too massive to run on low-power "edge" hardware, such as the embedded systems found in autonomous vehicles. Furthermore, these models are surprisingly fragile; environmental factors like fog, sensor noise, or motion blur can cause their accuracy to plummet.
Traditionally, these two issues are treated as separate engineering challenges. Compression techniques like pruning and quantization are used to reduce model size, while data augmentation is used to improve robustness. However, these goals often conflict: a highly compressed model may lack the redundancy required to handle noisy data, while a robust model may be too complex for real-time hardware.
Hybrid Corruption-Robustness focused Compression (HCRC)
To bridge this gap, researchers developed the Hybrid Corruption-Robustness focused Compression (HCRC) framework. HCRC is designed specifically for semantic segmentation—the task of labeling every pixel in an image—which is critical for safe automated driving. The method integrates four key pillars into a single optimization pipeline:
- Advanced Data Augmentation: Instead of simple filters, HCRC uses a complex mixing strategy. It samples multiple corruptions and mixes them using Dirichlet and Beta distributions, feeding both clean and "corrupted" versions to the network simultaneously.
- Consistency Loss: The framework employs a Jensen-Shannon Divergence (JSD) loss. This forces the network to produce consistent predictions for a clean image and its noisy counterpart, teaching the model that the underlying semantics remain the same despite visual noise.
- Iterative Magnitude Pruning: Rather than aggressive one-shot pruning, HCRC gradually removes the least important weights over several training cycles, allowing the model to adapt to its reduced capacity.
- In-Training Quantization (ITQ): Rounding errors are simulated during the training process itself. This allows the network to adjust its weights to accommodate lower 8-bit precision, rather than suffering accuracy loss from post-training conversion.
Key Insights and Results
The study evaluated HCRC using the DeepLabv3+ architecture on the Cityscapes and Sim KI-A benchmarks. Several critical insights emerged:
- Joint Optimization Advantage: The researchers proved that "Robustify then Compress" or "Compress then Robustify" sequences are less effective than the joint HCRC approach. Doing everything at once allows the pruning and quantization processes to account for robustness requirements from the start.
- Superior Robustness: On Cityscapes, HCRC improved the mean Performance under Corruption (mPC) by 8.39% absolute. Crucially, the model generalized to unseen corruptions, proving it learned a more general representation rather than just memorizing specific noise patterns.
- Significant Efficiency: The models remained effective even at 30% and 50% pruning ratios. When combined with 8-bit quantization, the total model size was reduced by approximately 80% (from ~237MB to ~47MB).
- ITQ vs. PTQ: In-Training Quantization (ITQ) was found to be vastly superior to Post-Training Quantization (PTQ), as it allows the model to "feel" the constraints of lower precision during the learning phase.
Conclusion and Future Directions
HCRC demonstrates that it is possible to create DNNs that are both leaner and meaner. By optimizing for size and resilience simultaneously, the framework provides a path toward safer, more efficient AI deployment in the real world. While current work focuses on common environmental corruptions, future research may expand this framework to address adversarial attacks and 3D object detection tasks on specialized automotive hardware.
Summary updated