Abstract
Model compression techniques allow to significantly reduce the computational cost associated with data processing by deep neural networks with only a minor decrease in average accuracy. Simultaneously, reducing the model size may have a large effect on noisy cases or objects belonging to less frequent classes. It is a crucial problem from the perspective of the models' safety, especially for object detection in the autonomous driving setting, which is considered in this work. It was shown in the paper that the sensitivity of compressed models to different distortion types is nuanced, and some of the corruptions are heavily impacted by the compression methods (i.e., additive noise), while others (blur effect) are only slightly affected. A common way to improve the robustness of models is to use data augmentation, which was confirmed to positively affect models' robustness, also for highly compressed models. It was further shown that while data imbalance methods brought only a slight increase in accuracy for the baseline model (without compression), the impact was more striking at higher compression rates for the structured pruning. Finally, methods for handling data imbalance brought a significant improvement of the pruned models' worst-detected class accuracy.
Introduction
In the deployment of autonomous driving systems, deep learning models must balance high-speed execution on edge hardware with stringent safety requirements. Model compression, specifically pruning, is a standard approach to achieve this efficiency by removing redundant weights or filters. However, standard evaluations often focus on "clean" datasets, overlooking how compression affects a model's robustness to real-world environmental noise and its bias against rare object classes. This research investigates the critical intersection of model sparsity, out-of-distribution (OOD) performance, and class imbalance.
Methodology and Datasets
The study utilizes a Faster R-CNN architecture, evaluating two pruning strategies: Unstructured Pruning (weight-level sparsity up to 95%) and Structured Pruning (filter-level sparsity up to 70%). To enhance robustness, the authors integrated naturalistic data augmentation (color distortion, blur, and noise) and specialized class-balancing techniques like Repeat Factor Sampling (RFS) and Inverse Squared Class Frequency (INV) weighting.
Performance was measured across several autonomous driving benchmarks:
- Cityscapes & BDD: General street scene detection.
- EuroCity Persons (ECP): Specifically used for day-to-night transition testing.
- Common Corruptions Benchmark: A synthetic suite of 15 distortion types including noise, weather, and digital artifacts.
Analysis of Robustness and Class Imbalance
Selective Sensitivity
Research findings indicate that compressed models exhibit "selective sensitivity." While they are remarkably vulnerable to additive noise (e.g., Gaussian or impulse noise), they remain relatively resilient to blur effects. This suggests that pruning primarily degrades the high-frequency components of a model's visual representation. By employing data augmentation, even highly compressed models can learn "texture-invariant" features, relying more on robust shapes than local textures.
The Pruning-Inequality Gap
Pruning does not impact all object classes equally. While common classes like cars remain stable, performance on "long-tail" classes (trucks, buses, trains) drops sharply as compression increases. Interestingly, class-balancing methods like INV weighting, which offer marginal gains for uncompressed models, show striking effectiveness for highly pruned models, significantly mitigating the disproportionate impact of sparsity on minority classes.
Key Results
- Robustness Recovery: On the Cityscapes Common Corruptions test, a 50% structurally pruned model's mAP on weather distortions improved from 0.105 to 0.243 through naturalistic augmentation.
- Class Balancing Gains: For a 70% pruned model on Cityscapes, INV balancing improved detection for the "Bus" class from 0.432 to 0.478 and the "Train" class from 0.1 to 0.184.
- Domain Adaptation: In day-to-night transitions (ECP dataset), pruned models showed a decrease in mAP comparable to uncompressed models, suggesting that pruning does not inherently compromise natural domain adaptation more than standard training.
Conclusion
While model compression introduces risks regarding noise sensitivity and class bias, these can be effectively mitigated. Naturalistic data augmentation and class-balancing techniques are essential tools for maintaining safety in compressed object detectors. Future research should explore the combined effects of pruning and quantization, as well as the impact on finer-grained subclasses within autonomous driving datasets.
Summary updated