TL;DR
This paper introduces "semantically stealthy" adversarial attacks for semantic segmentation models. Unlike previous attacks that often create obvious visual artifacts, these new attacks subtly manipulate target object labels (e.g., making a person disappear into the background) while preserving non-target labels, making them much harder to detect. This is achieved through an image-dependent perturbation generator and a novel regularizer that helps maintain semantic coherence.
What Problem Does This Paper Address?
Semantic segmentation models are crucial for understanding scenes in autonomous driving and robotics, but they are vulnerable to adversarial attacks. While many attacks exist, they often result in drastically altered segmentation maps that are easily identifiable as attacks. This makes them impractical for real-world scenarios where stealth is important. The challenge is to create adversarial attacks that can fool these models by manipulating specific object labels without introducing noticeable visual distortions or disrupting the segmentation of other objects in the scene.
Methods
The authors propose a framework for generating semantically stealthy adversarial attacks. The core idea is to use an image-dependent perturbation generator that learns to create perturbations tailored to each input image. This generator is trained alongside a regularizer, which is itself a segmentation model. The regularizer's role is to ensure that the features learned by the generator are semantically meaningful and can predict segmentation labels accurately, thus helping to preserve the non-targeted labels.
The attack framework involves:
- Generator: Takes an input image and produces a perturbation.
- Regularizer: Learns to segment the original image, ensuring semantic coherence.
- Target Model: The segmentation model to be attacked.
The attack manipulates target labels in three basic ways:
- Vanishing into context: Making a target object's label disappear by reclassifying its pixels into a semantically plausible background class.
- Adding fake labels: Introducing new, false labels for objects that are not present.
- Displacing target objects: A combination of removing a target object and adding a fake one elsewhere.
The loss function is designed to encourage the manipulation of target labels while minimizing changes to non-target labels, with a regularization term to maintain semantic consistency.
Datasets
The authors evaluate their approach on three widely used street-view datasets:
- Cityscapes: Contains urban street scenes with detailed semantic annotations.
- Mapillary Vistas Dataset: A large-scale dataset with diverse street-level imagery.
- BDD100K: A large and diverse dataset for autonomous driving research.
Key Insights and Ideas
- Semantic Coherence is Key to Stealth: The most significant insight is that preserving the semantic meaning of the scene, even while attacking specific labels, is crucial for making attacks stealthy. The regularizer plays a vital role in achieving this by ensuring the generator learns semantically relevant features.
- Image-Dependent Perturbations are More Effective: Unlike universal perturbations, generating attacks that are specific to each input image allows for more targeted and subtle manipulations.
- Attack Types Matter: The three proposed attack types (vanishing, adding fake, displacing) offer a flexible way to create diverse semantic attacks. The "vanishing into context" attack appears to be more challenging but potentially more stealthy.
- Generalization is Achievable: The framework demonstrates good generalization capabilities, meaning an attack model trained on one dataset can effectively attack models trained on different datasets, and vice-versa.
- Parameter-wise Efficiency is a New Metric: The paper introduces the concept of "attack efficiency" (ratio of attack model size to target model size), highlighting that smaller, more efficient attack models can still achieve significant success rates.
Key Results
- The proposed semantically stealthy attacks achieve relatively high success rates on Cityscapes, Mapillary, and BDD100K datasets.
- For Type #1 (vanishing), manipulated rates range from 66.56% to 74.71% while preserving non-targeted labels at 91.40% to 97.09%.
- For Type #2 (adding fake), manipulated rates are higher (68.69% to 91.27%) but preserving non-targeted labels is more challenging (88.62% to 92.76%).
- Type #3 (combining attacks) is the most difficult and shows lower overall performance.
- The regularizer improves the manipulated rate by around 8-12% across datasets.
- The framework shows good generalization across datasets and target models, with success rates often above 90% when attacking models trained on different distributions.
- Smaller attack models (e.g., 4% of target model size) can still achieve around 70% manipulated rate.
Limitations and Future Work
- Residuals in Type #1: The paper acknowledges that Type #1 attacks can sometimes leave "residuals" or faint traces of the original object's presence.
- Difficulty of Type #3: Combining attacks (Type #3) is shown to be the most challenging, leading to a decrease in overall performance.
- Computational Cost: While the framework is more efficient than some prior methods, training large segmentation models and generators still requires significant computational resources.
- Future Work: The authors suggest exploring other high-level semantic manipulations by combining the basic attack types and further investigating parameter-wise efficiency for more resource-constrained scenarios. They also propose exploring the role of the regularizer in generating labels that come from the target model.
Summary updated