Abstract
Adversarial attacks on semantic segmentation models introduce structured, periodic artifacts in the frequency domain that are characteristic of the model's architecture. This paper proposes a lightweight defense mechanism using Wiener filtering to suppress these adversarial frequencies. By estimating the power spectrum of perturbations, the defense acts as a preprocessing step that improves robustness across various attack types without requiring model retraining. Evaluations on the Cityscapes dataset demonstrate significant gains in mean Intersection over Union (mIoU) and structural similarity while maintaining high performance on clean images.
Introduction
Deep neural networks (DNNs) are notoriously vulnerable to adversarial perturbations—tiny, often imperceptible changes to an image that cause a model to fail. While many defenses exist for simple image classification, they often fail to scale to complex, high-resolution tasks like semantic segmentation. Existing defenses usually rely on adversarial training, which is computationally grueling and often fails to generalize to new types of attacks. This research seeks a more fundamental understanding: what do these perturbations actually look like in the frequency domain, and can we remove them before they reach the model?
Frequency-Domain Artifacts as Architectural Fingerprints
Analysis using the Discrete Fourier Transform (DFT) reveals that adversarial attacks on semantic segmentation models leave distinct, "fingerprint-like" artifacts in the frequency domain. These patterns are primarily tied to the model's architecture—specifically its internal interpolation layers (like bilinear or bicubic upsampling)—rather than the specific image content or the mathematical method used to generate the attack.
For example, models like ICNet show distinct "grid-shaped" artifacts in their frequency spectra. Because these artifacts are tied to the architecture's internal structure, the frequency patterns remain remarkably consistent across different attacks, suggesting an opportunity for an attack-agnostic defense.
Methodology: Wiener Filter Defense
The proposed defense utilizes a Wiener filter, a classical signal processing tool designed to minimize the mean square error between a noisy signal and its original version. The defense operates in two stages:
- Stage 1 (Training): The average power spectrum of adversarial perturbations is computed across a training set to create a "combined" Wiener filter specific to a model architecture.
- Stage 2 (Inference): This filter is applied as a simple preprocessing step. Any incoming image is converted to the frequency domain, multiplied by the filter to suppress adversarial frequencies, and converted back to the spatial domain before being fed into the segmentation model.
Experimental Results
The study primarily used the Cityscapes dataset to evaluate the defense against four types of attacks (mFGSM, Metzen, IM, and Mopuri) across two different model architectures: ICNet and FCN.
- Visual Fidelity: The Wiener filter significantly improved the Structural Similarity Index (SSIM) of attacked images (e.g., from 0.79 to 0.91) while keeping clean images virtually untouched (only a 0.5% drop in performance).
- Robustness Gains: On the ICNet model, the defense improved the mean Intersection over Union (mIoU) by roughly 6-8% absolute across various attacks.
- Generalization: A filter trained on one type of attack was still highly effective against completely different, "unseen" attacks, confirming the architecture-dependent nature of the perturbations.
- State-of-the-Art Comparison: The proposed method outperformed five common denoising-based defenses, including JPEG compression and Non-Local Means (NLM) smoothing.
Conclusion and Future Work
Wiener filtering provides a computationally efficient, "plug-and-play" defense for semantic segmentation that requires zero changes to the underlying neural network. While effective, the authors note that a "white-box" attacker who knows the defense is in place can use techniques like Backward Pass Differential Approximation (BPDA) to attempt to bypass the filter. Future research could explore more sophisticated frequency-domain modeling or combining this lightweight preprocessing with other defense layers to create a "defense-in-depth" strategy for safety-critical applications like autonomous driving.
Summary updated