TL;DR
This paper introduces SADL, a novel unsupervised method to detect and pinpoint adversarial patch attacks on optical flow networks. By analyzing feature activations and applying spatial constraints, SADL effectively identifies and localizes these attacks without needing prior knowledge or training data specific to the attacks, offering a crucial step towards more robust computer vision systems.
What Problem Does This Paper Address?
Optical flow estimation, a fundamental task in computer vision used in applications like self-driving cars and robotics, has seen significant advancements with deep learning. However, these powerful networks are vulnerable to adversarial attacks, specifically "patch attacks," where small, crafted image patches are inserted to manipulate the estimated flow. These attacks can have serious security implications, leading to incorrect motion predictions. While previous work has demonstrated the existence and impact of these attacks, there's been a lack of methods to reliably detect and, importantly, localize them in an unsupervised manner, meaning without needing to train a separate model on known attack types. This paper aims to fill that gap by providing a way to identify and pinpoint these malicious patches within the network's internal representations.
Methods
The core of SADL is a Spatially Constrained Subset Scanning approach. Here's a breakdown:
- Feature Extraction: SADL works by examining the activations of nodes in the inner layers of pre-trained optical flow networks.
- p-value Computation: For each node, it compares the activation of a test image pair against a distribution of activations from clean (non-attacked) image pairs. This comparison yields a "p-value" for each node, indicating how unusual its activation is. If the network is attacked, these p-values are expected to deviate from a uniform distribution.
- Non-parametric Scan Statistics (NPSS): SADL uses a statistical measure (specifically, a variant of the Berk-Jones statistic) to quantify how "non-uniform" the distribution of p-values is across a subset of nodes. A higher score indicates a greater deviation from the expected, suggesting an anomaly.
- Spatial Constraints: To avoid detecting anomalies that are spatially scattered and not indicative of a localized patch attack, SADL introduces two key constraints:
- Spatial Alignment: It ensures that when evaluating a spatial location, activations from all channels at that location are considered together.
- Proximity Constraint: It limits the search for anomalous subsets to a small, local spatial neighborhood (e.g., a k x k region), making it more likely to find a contiguous region of anomalous features corresponding to a patch.
- Spatial-Channel Optimization (SCO): This further refines the process by allowing the method to select not only a spatial subset of anomalous nodes but also a subset of channels within that spatial region, making the detection more precise.
- Localization: Once an anomalous subset of features is detected, SADL maps these back to the input image space to pinpoint the likely location of the adversarial patch.
Crucially, SADL is unsupervised and requires no prior knowledge of the attack type or dedicated training.
Datasets
The paper evaluates SADL on several standard optical flow datasets:
- KITTI 2015: A widely used dataset for autonomous driving, providing labeled optical flow.
- MPI-Sintel: A dataset featuring computer-animated scenes with ground truth flow.
- Raw KITTI and Raw Sintel: These datasets, lacking ground truth flow, are used for generating background image pairs to establish the "normal" activation distributions.
Key Insights and Ideas
- Inner Layer Analysis is Key: Adversarial attacks, even if small, propagate and amplify through the network's layers. Analyzing these internal representations, rather than just the final output, is crucial for detection.
- Spatial Locality Matters for Attacks: Patch attacks are inherently localized. SADL's success hinges on exploiting this spatial characteristic by constraining the search for anomalies to local regions.
- Unsupervised Detection is Feasible: The paper demonstrates that it's possible to detect and localize attacks without needing to train a specific classifier for each attack type, which is a significant advantage for real-world deployment.
- Different Networks Exhibit Different Vulnerabilities: The study shows that some networks (like FlowNetC) amplify attack effects more in deeper layers, while others (like RAFT) might be more robust or show different patterns.
Key Results
- High Detection Performance: SADL achieves high Area Under the Curve (AUC) scores for attack detection, outperforming a baseline subset scanning method without spatial constraints. For example, on KITTI 2015 with a 153x153 patch attack on RAFT, SADL achieves an AUC of 1.00.
- Effective Localization: The method demonstrates strong performance in localizing attacks, with average precision (AP) and average recall (AR) scores often exceeding 0.90 for larger patches. For instance, on KITTI 2015 with a 153x153 patch attack on RAFT, AP is 1.00 and AR is 0.61.
- Robustness to Patch Size: SADL shows good performance across different patch sizes, though localization accuracy generally decreases with smaller patches.
- Layer-wise Performance: Detection performance varies across network layers. For FlowNetC, detection improves in deeper layers, suggesting attack amplification. For other networks, earlier layers might be more informative.
Limitations and Future Work
- Smaller Patch Localization: SADL sometimes struggles to localize very small patch attacks (e.g., 51x51), which have a less pronounced effect on flow estimation anyway.
- Hyperparameter Tuning: The paper notes that performance can be further improved by optimizing hyperparameters like the size of the local region (k) for the proximity constraint.
- Future Directions: The authors suggest using the detected and localized attacks to develop mitigation techniques for optical flow estimators. Further analysis into why performance differs across networks and datasets is also proposed.
Summary updated