Abstract
Convolutional neural networks have recently advanced the state of the art in many tasks including edge and object boundary detection. However, in this paper, we demonstrate that these edge detectors inherit a troubling property of neural networks: they can be fooled by adversarial examples. We show that adding small perturbations to an image causes HED, a CNN-based edge detection model, to fail to locate edges, to detect nonexistent edges, and even to hallucinate arbitrary configurations of edges. More surprisingly, we find that these adversarial examples transfer to other CNN-based vision models. In particular, attacks on edge detection result in significant drops in accuracy in models trained to perform unrelated, high-level tasks like image classification and semantic segmentation.
Introduction
Adversarial examples—inputs specifically designed to trick machine learning models—have been a major focus in high-level computer vision tasks like image classification. However, low-level vision tasks, such as edge detection, were long assumed to be more robust due to their perceived simplicity. This paper challenges that assumption, demonstrating that state-of-the-art edge detectors like Holistically-Nested Edge Detection (HED) are highly susceptible to adversarial perturbations. Crucially, the authors investigate whether these "broken" low-level understandings propagate to and impair high-level vision systems.
Methodology and Attack Framework
The researchers adapted the Momentum Iterative Fast Gradient Sign Method (MI-FGSM) to target the HED model, which uses a VGG16 backbone to produce edge maps at multiple scales. To effectively manipulate these outputs, the authors introduced several attack variants:
- Suppression: Forcing the model to ignore existing edges.
- Activation (Hallucination): Forcing the model to detect edges where none exist.
- Inverse-target: A combination where real edges are suppressed and fake ones are activated.
A key technical finding was the necessity of adjusting the loss function. While standard HED uses class-balanced loss to handle the sparsity of edge pixels, the authors found that a 1:1 class weighting was required to successfully "hallucinate" complex edge patterns like text, as shown in the figure above.
Key Insights and Transferability
The most significant finding is the transferability of these low-level attacks. Perturbations designed to fool an edge detector effectively "blind" high-level models, suggesting that these models rely on edge-like features internally even if not explicitly trained for edge detection.
- Hallucination vs. Suppression: It is significantly easier to create fake edges than to hide real, high-contrast ones.
- Layer Vulnerability: Attacking deeper layers of the HED model, which have larger receptive fields, proved more effective than targeting early layers.
- Shape Bias: Models trained with a "shape bias" (e.g., using Stylized-ImageNet) are more robust to edge suppression but remain highly vulnerable to untargeted edge attacks, as the perturbations target the very features these models prioritize.
Results
The study utilized benchmarks including BSDS500, ImageNet, and PASCAL VOC 2012. Key results include:
- Edge Detection Failure: On BSDS500, HED's performance (ODS F-score) plummeted from 0.775 to 0.312 under Inverse-target attacks.
- Classification Transfer: Attacks on HED caused VGG16's ImageNet accuracy to drop from 71.2% to 14.3%. Even the robust ResNet152 saw a drop from 78% to 43.7%.
- Segmentation Transfer: For DeepLabv3+, the mean Intersection over Union (mIOU) dropped from 0.822 to 0.553 when facing edge-activation attacks.
Conclusion
This work proves that adversarial vulnerability extends to fundamental low-level vision. The fact that these attacks transfer across tasks highlights a systemic weakness in current CNN architectures. Future research must focus on developing robust defenses and detection mechanisms for these low-level adversarial threats, as standard defenses like adversarial training remain computationally expensive for dense, high-dimensional tasks.
Code for this project will be made public.
Summary updated