Abstract
Testing autonomous Cyber-Physical Systems (CPS) is challenging due to the vast state space of visual inputs. This work proposes a novel fuzzing method that utilizes structured patterns derived from convolutional neural network (CNN) filters to stress-test vision-based autonomous systems. Experimental results demonstrate that this approach identifies safety-critical failures significantly faster than traditional random noise injection, achieving speedups of up to 1610% in discovering counterexamples.
Introduction
Autonomous systems, ranging from self-driving cars to automated surveillance, increasingly rely on high-resolution visual sensors. Ensuring the safety and reliability of these systems is difficult because their input state space is effectively infinite. Traditional software testing often employs "fuzzing" — the injection of random data to trigger crashes. However, in the context of computer vision, simple random pixel noise is frequently ignored by robust machine learning models or filtered out as "salt and pepper" noise. To uncover truly dangerous edge cases, perturbations must be "meaningful" to the underlying vision algorithms.
Methodology
The proposed method bridges the gap between random noise and sophisticated adversarial attacks by using structured features.
Feature Extraction and Perturbation
Instead of random pixels, the researchers extract patterns from the convolutional filters of a pre-trained, unrelated CNN. These filters represent shapes and textures that neural networks are naturally tuned to detect. These patterns are normalized and added to random locations within an image or video frame.
Statistical Validation
To ensure rigorous testing, the authors employ Wald’s Sequential Probability Ratio Test (SPRT). This allows the system to statistically determine the number of samples required to confirm whether a vision algorithm meets a specific safety threshold (e.g., 99.9999% accuracy).
Experimental Evaluation
The method was evaluated in two primary environments: the OpenCV library (testing the Histogram of Oriented Gradients (HOG) human detection algorithm) and the Udacity Open-Source Simulator (a self-driving car environment).
Key Results
The CNN-based fuzzing approach consistently outperformed random perturbations in terms of speed and effectiveness. In one instance, the CNN-based approach found a counterexample in 53 seconds, compared to 855 seconds for random noise — a 1610% speedup.
| Image Test | Random Perturbation (s) | CNN Perturbation (s) | Speedup |
|---|---|---|---|
| Case 1 | 74.72 | 45.24 | 165% |
| Case 2 | 855.24 | 53.13 | 1610% |
| Case 3 | 84.00 | 55.44 | 151% |
Discussion
A key insight is that structured noise is much harder for vision systems to ignore than random noise. Furthermore, the researchers found that perturbing seemingly irrelevant areas — such as the periphery of a camera frame — could cause significant failures, such as an autonomous vehicle deviating and crashing. This highlights the non-intuitive nature of failure modes in deep learning systems. Crucially, features that are "interesting" to one CNN are often disruptive to another, suggesting a cross-model vulnerability.
Conclusion and Future Work
This work-in-progress demonstrates that using CNN-derived features for fuzzing is a highly efficient way to identify vulnerabilities in autonomous systems. Future work will explore optimizing the selection of specific filter types for different autonomous tasks and investigating defensive applications, such as training models on these perturbations to enhance robustness.
Summary updated